Bug #1770
open
Suricata takes very long time to start using hyperscan and large/custom detect settings
Added by Peter Manev over 8 years ago.
Updated over 5 years ago.
Description
Using 3.1dev (rev b92a08b) with the following settings -
detect:
profile: custom
custom-values:
toclient-groups: 600
toserver-groups: 800
sgh-mpm-context: full
and
mpm-algo: hs
it takes very long time for Suricata to start as opposed to using the same settings but with
mpm-algo: ac-ks
profile: low/medium/high
are working fine though.
mpm-algo: hs
detect.profile = high
detect.sgh-mpm-context: full
also takes much longer (though not as long as detect.profile = custom) than mpm-algo: ac/mpm-algo: ac-ks
- Assignee set to Anonymous
- Target version set to TBD
- Assignee set to Community Ticket
- Status changed from New to Closed
With Hyperscan yes (though not as long as initially reported) -
suricata -i eno1 --set "detect.profile = high" --set "detect.sgh-mpm-context = full"
[12576] 30/7/2019 -- 15:10:31 - (suricata.c:1071) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (3a912446a 2019-07-22) running in SYSTEM mode
....
[12576] 30/7/2019 -- 15:13:58 - (tm-threads.c:2145) <Notice> (TmThreadWaitOnThreadInit) -- all 112 packet processing threads, 10 management threads initialized, engine started.
- Status changed from Closed to Feedback
Does this increase with the amount of rules as well?
Yes - if you specify 0 rules it will load faster.
what system is this? I also see quite a long load time around 30-40s sometimes but with over 35k rules.
Any Debian and Ubuntu with the following settings in yaml -
detect:
profile: custom
custom-values:
toclient-groups: 600
toserver-groups: 800
sgh-mpm-context: full
mpm-algo: hs
It is not seconds but minutes it needs.
Andreas - are you able to reproduce this ?
I meant more in regards to hardware :)
I see a rather big amount of threads, can you check if it changes if you change the threads amount?
No , not related to HW/number of threads in my case.
Can you please confirm ?
I did check again in detail and confirm that it's taking very long:
22/8/2019 -- 07:45:16 - <Notice> - This is Suricata version 4.1.4 RELEASE
22/8/2019 -- 08:10:01 - <Notice> - all 16 packet processing threads, 6 management threads initialized, engine started.
Seems to increase a lot with more rules :)
Thank you for confirming.
I think the combination
detect:
profile: custom
custom-values:
toclient-groups: 600
toserver-groups: 800
sgh-mpm-context: full
is excessive but even with profile: high when using mpm-algo hyperscan the load times are high - good few minutes with a full ruleset.
Also available in: Atom
PDF