Optimization #2803

test phase should use less memory

Added by Victor Julien 6 months ago. Updated 6 months ago.

Target version:


While suricata-update runs it's test phase, its memory use remains high. This seems unnecessary, as all the rule parsing & sorting is already done. The only result of the test is pass or fail, no complicated rule actions should be done anymore.



Updated by Jason Ish 6 months ago

2 things come to mind to immediately solve this. Set some of the dict's that are storing all the state to None and see if they get garbage collected. Or break the big main() up into smaller functions, which is more likely to garbage collect stuff as they get reference counted to 0 on function return, plus main() could probably benefit some breaking it up a bit.


Updated by Victor Julien 6 months ago

I should add that I only tested with python2. The way I observed this is simple: htop showed the mem use of suricata-update while the test command (a suricata -T variant) was running. It can probably be simulated by simply having a test script that sleeps for some time.


Updated by Jason Ish 6 months ago

  • Status changed from Assigned to Closed
  • Target version changed from 1.1.0 to 1.0.4

Fixed by setting some collection types to None before running the test.

This is probably about the best we can do without some redesign of suricata-update internals.

Also available in: Atom PDF