tracking: memory fragmentation
During start up (mostly detection engine init) we alloc and free enormous amounts of small memory blocks. This leads to memory fragmentation.
This task is a place holder for tracking that. Addressing all/most of this is going to be a long term effort.
Updated by Gianni Tedesco 4 months ago
Actually the main culprit of heap fragmentation is reassembly. Keep doing increasing realloc() as we do for tcp stream reassembly is worst-case behaviour, and it can cause several gigabytes of excess heap to be used up. The only way to get it back is to attach gdb and call malloc_trim(0).
Reassembly buffers need to be a chain of fixed size blocks to avoid this situation. You can copy the blocks out to the final (per-thread) buffer at the end, just before injecting the packet for inspection.