Optimization #8484
closedThreshold hash table doesn't hash IPv4 addresses, resulting in poor bucket distribution
Description
Hello. I have a signinficant performance degradation with suricata 8 compared to 7.0.15. Judging by flamegraphs suricata spends much more time in DetectEngineRunPostRules/PacketAlertFinalize/ThresholdGetFromHash. My threshold rules are track-by-both and network is IPv4 only.
Examining resulting hash table in detect-engine-threshold.c ctx (shouldn't it be static if it uses so non-specific name?) I could see only one bucket holding all the values, so naturally linear search over linked list is very costly. My test iterates IPs over two networks so resulting bucket number is always the same, even though IPs are different.
In 7.0.15 ippair hashed both IPs, but in suricata8 it just adds IP without hashing it https://github.com/OISF/suricata/blob/be36e67f52fd6b6b104cff9ceb38369dbaf0eaaf/src/detect-engine-threshold.c#L266 . Adding hashword here fixed the problem for me.
PA Updated by Philippe Antoine 16 days ago ยท Edited
- Status changed from New to Closed
- Target version changed from TBD to 9.0.0-beta1
Fixed by c27868a8c59b27cd25d5b281337272d283fcb247 in #8617
PA Updated by Philippe Antoine 16 days ago
- Related to Bug #8617: threshold: seed only partially applied in IPv6 hash added