Optimization #1603
closedIncrease STATE_QUEUE_CONTAINER_SIZE to manage huge rulesets
Description
If you have a huge ruleset (>100k rules) with unique patterns you might run into the following error:
<Critical> - [ERRCODE: SC_ERR_AHO_CORASICK(174)] - Just ran out of space in the queue. Fatal Error. Exiting. Please file a bug report on this
Which is a result of the STATE_QUEUE_CONTAINER_SIZE defined to 65536. If i increase the value it works. So unless there is another good reason to restrict it to that value i would suggest to increase it.
Updated by Andreas Herz over 8 years ago
Any suggestion to which value we should set it?
Updated by Victor Julien over 8 years ago
What (if any) are the potential side effects?
Updated by Andreas Herz about 8 years ago
StateQueue consuming lots of memory :) in theory we could jump to 4.294.967.296 at least but i will play around with it a bit and see if i can find any sideffects.
Updated by Andreas Herz about 8 years ago
Unless we switch to SCMalloc() and SCFree() instead of using the stack, we're limited and 65536 is a solid value. Since Victor also found some 16bit limitiations i would recommend to discuss some higher value that won't exceed memory usage too much which might happen if we go for the maximum possible.
Updated by Andreas Herz over 7 years ago
- Status changed from New to Closed
So far there is no need for that anymore from my side, so for now we can close it