Project

General

Profile

Actions

Bug #4685

closed

detect: too many prefilter engines lead to FNs

Added by Victor Julien about 1 year ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:
Needs backport to 5.0, Needs backport to 6.0

Description

Fix for QA ticket #4672

Bad int handling leads to missed inspection when prefilter engine list gets too long.


Files

test.rules (9.93 KB) test.rules Victor Julien, 09/15/2021 08:14 AM

Related issues 2 (0 open2 closed)

Copied to Bug #4687: detect: too many prefilter engines lead to FNsClosedVictor JulienActions
Copied to Bug #4688: detect: too many prefilter engines lead to FNsClosedVictor JulienActions
Actions #1

Updated by Victor Julien about 1 year ago

This is turning out to be a more complex issue. While the initial analysis is correct for a full ET ruleset, the issue is actually worse if a ruleset uses more keywords. Especially if transforms are added, its easy to exceed the 63 bits (1 is reserved) used for tracking which tx prefilter engines have run.

I'm attaching a example rulefile that leads to the local_id getting to 160 in 6.0.x and 100 in 5.0.x. The difference is due to duplicate engines for http1/http2 in 6.0.x.

Some thoughts on addressing this:
- the most direct way of fixing this is to expand the bit space. However this will be fairly complex due to how the data structure is ultimately in AppLayerTxData in Rust.
- perhaps we can track not per engine, but per 'progress' value. At a certain progress value all relevant engines should run so maybe it would be sufficient to track what ran based on this value.
- not really a solution but still a thing to state: we need to untangle the engines list by alproto somehow, as currently we skip most engines in the list due to them not being for our alproto.

Actions #2

Updated by Jeff Lucovsky about 1 year ago

  • Copied to Bug #4687: detect: too many prefilter engines lead to FNs added
Actions #3

Updated by Jeff Lucovsky about 1 year ago

  • Copied to Bug #4688: detect: too many prefilter engines lead to FNs added
Actions #4

Updated by Jason Ish about 1 year ago

  • Affected Versions 5.0.6, 6.0.3 added
Actions #6

Updated by Victor Julien 10 months ago

  • Private changed from Yes to No
Actions

Also available in: Atom PDF