Bug #1770
openSuricata takes very long time to start using hyperscan and large/custom detect settings
Description
Using 3.1dev (rev b92a08b) with the following settings -
detect: profile: custom custom-values: toclient-groups: 600 toserver-groups: 800 sgh-mpm-context: full
and
mpm-algo: hs
it takes very long time for Suricata to start as opposed to using the same settings but with
mpm-algo: ac-ks
profile: low/medium/high
are working fine though.
Updated by Peter Manev about 8 years ago
mpm-algo: hs detect.profile = high detect.sgh-mpm-context: full
also takes much longer (though not as long as detect.profile = custom) than mpm-algo: ac/mpm-algo: ac-ks
Updated by Andreas Herz almost 8 years ago
- Assignee set to Anonymous
- Target version set to TBD
Updated by Peter Manev about 5 years ago
- Status changed from New to Closed
With Hyperscan yes (though not as long as initially reported) -
suricata -i eno1 --set "detect.profile = high" --set "detect.sgh-mpm-context = full" [12576] 30/7/2019 -- 15:10:31 - (suricata.c:1071) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (3a912446a 2019-07-22) running in SYSTEM mode .... [12576] 30/7/2019 -- 15:13:58 - (tm-threads.c:2145) <Notice> (TmThreadWaitOnThreadInit) -- all 112 packet processing threads, 10 management threads initialized, engine started.
Updated by Andreas Herz about 5 years ago
Does this increase with the amount of rules as well?
Updated by Peter Manev about 5 years ago
Yes - if you specify 0 rules it will load faster.
Updated by Andreas Herz about 5 years ago
what system is this? I also see quite a long load time around 30-40s sometimes but with over 35k rules.
Updated by Peter Manev about 5 years ago
Any Debian and Ubuntu with the following settings in yaml -
detect: profile: custom custom-values: toclient-groups: 600 toserver-groups: 800 sgh-mpm-context: full
mpm-algo: hs
It is not seconds but minutes it needs.
Andreas - are you able to reproduce this ?
Updated by Andreas Herz about 5 years ago
I meant more in regards to hardware :)
I see a rather big amount of threads, can you check if it changes if you change the threads amount?
Updated by Peter Manev about 5 years ago
No , not related to HW/number of threads in my case.
Can you please confirm ?
Updated by Andreas Herz about 5 years ago
I did check again in detail and confirm that it's taking very long:
22/8/2019 -- 07:45:16 - <Notice> - This is Suricata version 4.1.4 RELEASE 22/8/2019 -- 08:10:01 - <Notice> - all 16 packet processing threads, 6 management threads initialized, engine started.
Seems to increase a lot with more rules :)
Updated by Peter Manev about 5 years ago
Thank you for confirming.
I think the combination
detect: profile: custom custom-values: toclient-groups: 600 toserver-groups: 800 sgh-mpm-context: full
is excessive but even with profile: high when using mpm-algo hyperscan the load times are high - good few minutes with a full ruleset.