Project

General

Profile

Actions
Copy link

Bug #1770

open

Suricata takes very long time to start using hyperscan and large/custom detect settings

Added by Peter Manev almost 8 years ago. Updated over 4 years ago.

Status:
Feedback
Priority:
Normal
Assignee:
Community Ticket
Target version:
TBD
Affected Versions:
Effort:
Difficulty:
Label:

Description

Using 3.1dev (rev b92a08b) with the following settings -

detect:
  profile: custom
  custom-values:
    toclient-groups: 600
    toserver-groups: 800
  sgh-mpm-context: full

and 

mpm-algo: hs

it takes very long time for Suricata to start as opposed to using the same settings but with 
mpm-algo: ac-ks


  profile: low/medium/high

are working fine though.
Actions
Copy link
 #1

Updated by Peter Manev almost 8 years ago 

mpm-algo: hs
detect.profile = high
detect.sgh-mpm-context: full

also takes much longer (though not as long as detect.profile = custom) than mpm-algo: ac/mpm-algo: ac-ks

Actions
Copy link
 #2

Updated by Andreas Herz over 7 years ago

  • Assignee set to Anonymous
  • Target version set to TBD
Actions
Copy link
 #3

Updated by Andreas Herz about 5 years ago

  • Assignee set to Community Ticket
Actions
Copy link
 #4

Updated by Andreas Herz over 4 years ago

Is this still an issue?

Actions
Copy link
 #5

Updated by Peter Manev over 4 years ago

  • Status changed from New to Closed

With Hyperscan yes (though not as long as initially reported) - 

suricata -i eno1 --set "detect.profile = high"  --set "detect.sgh-mpm-context = full" 
[12576] 30/7/2019 -- 15:10:31 - (suricata.c:1071) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (3a912446a 2019-07-22) running in SYSTEM mode
....

[12576] 30/7/2019 -- 15:13:58 - (tm-threads.c:2145) <Notice> (TmThreadWaitOnThreadInit) -- all 112 packet processing threads, 10 management threads initialized, engine started.

Actions
Copy link
 #6

Updated by Peter Manev over 4 years ago

  • Status changed from Closed to Feedback
Actions
Copy link
 #7

Updated by Andreas Herz over 4 years ago

Does this increase with the amount of rules as well?

Actions
Copy link
 #8

Updated by Peter Manev over 4 years ago

Yes - if you specify 0 rules it will load faster.

Actions
Copy link
 #9

Updated by Andreas Herz over 4 years ago

what system is this? I also see quite a long load time around 30-40s sometimes but with over 35k rules.

Actions
Copy link
 #10

Updated by Peter Manev over 4 years ago

Any Debian and Ubuntu with the following settings in yaml -

detect:
  profile: custom
  custom-values:
    toclient-groups: 600
    toserver-groups: 800
  sgh-mpm-context: full

mpm-algo: hs

It is not seconds but minutes it needs.
Andreas - are you able to reproduce this ?

Actions
Copy link
 #11

Updated by Andreas Herz over 4 years ago

I meant more in regards to hardware :)

I see a rather big amount of threads, can you check if it changes if you change the threads amount?

Actions
Copy link
 #12

Updated by Peter Manev over 4 years ago

No , not related to HW/number of threads in my case.
Can you please confirm ?

Actions
Copy link
 #13

Updated by Andreas Herz over 4 years ago

I did check again in detail and confirm that it's taking very long:

22/8/2019 -- 07:45:16 - <Notice> - This is Suricata version 4.1.4 RELEASE
22/8/2019 -- 08:10:01 - <Notice> - all 16 packet processing threads, 6 management threads initialized, engine started.

Seems to increase a lot with more rules :)

Actions
Copy link
 #14

Updated by Peter Manev over 4 years ago

Thank you for confirming.
I think the combination

detect:
  profile: custom
  custom-values:
    toclient-groups: 600
    toserver-groups: 800
  sgh-mpm-context: full

is excessive but even with profile: high when using mpm-algo hyperscan the load times are high - good few minutes with a full ruleset.

Actions
Copy link

Also available in: Atom PDF