App-Layer: Not all parsers register TX detect flags that should
User reported slowdown in the DHCP parser. It turns out that the DHCP parser did not register tx detect flag callbacks resulting the transactions on a flow never being freed. This is particular noticeable on DHCP due to the 0.0.0.0->255.255.255.255 address pair that is used for many DHCP requests.
First fixed as an app-layer fixup to handle parsers without tx detect flags, such as DHCP that don't need them as they don't register any detection engines.
The second fix was to throw a fatal error if a detect engine is registered for an app-layer proto without tx detect flags. This should never happen in production, its more of a check during development.
Finally protocols that were missing tx detect flags that needed them were fixed up to use them.
Updated by Jason Ish about 2 years ago
Victor Julien wrote:
We should also review if any other parser has the same issue. Perhaps the API should enforce it.
Looks like most of the Rust ones (but not all).
Is there any parser where it would make sense to not have these callbacks set?