Bug #4499
openSudden and enormous memory leak
Description
Hi,
We have a large installed base of suricata 6.0.2. We're seeing a memory leak that can consume unbounded amounts of memory (even hundreds of gigabytes over a few days). The leak appears suddenly in bursts, not gradually over time. This is affecting between 15% and 20% of all our suricata instances.
We cannot find anything in common between the leaky instances other than that they all see the following protocols: TLS, DNS, HTTP and DCE-RPC.
Attached a graph showing various telemetry memuse stats over time alongside resident set size.
We were seeing the issue also in 6.0.1, and we think we were also seeing it with 6.0.0 judging by some OOM kernel panics we saw...
Files
Updated by Gianni Tedesco over 3 years ago
- File build.txt build.txt added
- File config.txt config.txt added
Updated by Gianni Tedesco over 3 years ago
We've ruled out DCE-RPC by disabling it and confirming that the leak still happens.
Updated by Gianni Tedesco over 3 years ago
- File malloc-trim.png malloc-trim.png added
We've ruled out memory fragmentation by using gdb to attach to suricata and calling malloc_trim(0).
Updated by Victor Julien over 3 years ago
Can you share stats.log entries to see if they hold any clues?
Updated by Victor Julien over 1 year ago
@Gianni Tedesco are you still seeing this issue?
Updated by Philippe Antoine 5 months ago
- Status changed from New to Feedback
- Target version set to TBD
Updated by Gianni Tedesco about 1 month ago ยท Edited
I think the issue went away with some upgrade along the way. Either that or changes to config or rules (unintentionally) solved it.