Optimization #1603: Increase STATE_QUEUE_CONTAINER_SIZE to manage huge rulesets - Suricata - Open Information Security Foundation

Actions

Copy link

Optimization #1603

closed

Increase STATE_QUEUE_CONTAINER_SIZE to manage huge rulesets

Added by Andreas Herz almost 10 years ago. Updated almost 8 years ago.

Status:

Closed

Priority:

Low

Assignee:

Andreas Herz

Target version:

Effort:

Difficulty:

Label:

Description

If you have a huge ruleset (>100k rules) with unique patterns you might run into the following error:

<Critical> - [ERRCODE: SC_ERR_AHO_CORASICK(174)] - Just ran out of space in the queue.  Fatal Error.  Exiting.  Please file a bug report on this

Which is a result of the STATE_QUEUE_CONTAINER_SIZE defined to 65536. If i increase the value it works. So unless there is another good reason to restrict it to that value i would suggest to increase it.

Actions

Copy link

Updated by Andreas Herz almost 10 years ago

Assignee set to Andreas Herz

Actions

Copy link

Updated by Andreas Herz almost 10 years ago

Any suggestion to which value we should set it?

Actions

Copy link

Updated by Victor Julien almost 10 years ago

What (if any) are the potential side effects?

Actions

Copy link

Updated by Andreas Herz over 9 years ago

StateQueue consuming lots of memory :) in theory we could jump to 4.294.967.296 at least but i will play around with it a bit and see if i can find any sideffects.

Actions

Copy link

Updated by Andreas Herz over 9 years ago

Unless we switch to SCMalloc() and SCFree() instead of using the stack, we're limited and 65536 is a solid value. Since Victor also found some 16bit limitiations i would recommend to discuss some higher value that won't exceed memory usage too much which might happen if we go for the maximum possible.

Actions

Copy link