Project

General

Profile

Actions

Optimization #580

closed

use mpm results for secondary patterns if available

Added by Victor Julien over 11 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Low
Assignee:
-
Target version:
-
Effort:
Difficulty:
Label:

Description

While reviewing mpm pattern distribution, I noticed quite a few sigs have patterns that are not the sigs fast_pattern that are in the mpm anyway because of other sigs. In these cases we could possibly use the mpm results in content validation.

Actions #1

Updated by Victor Julien about 11 years ago

Further explanation of this idea. Say we have 2 sigs in a sgh:

content:"abc"; sid:1;
content:"reallylongstring"; content:"abc"; sid:2;

And assume that for sid 2 we use "reallylongstring" in the mpm.

When the mpm indicates a potential match for sid 2, we know that the mpm actually also looked for sid 2's other pattern as that is the mpm pattern for sid 1. So we may be able to use this info to determine if sid2 can match. Instead of looking for "abc" with the spm, we can lookup in the pmq if we have a match for "abc".

Obviously, this wouldn't be enough when there is a complex relationship between sid 2's patterns, but it may still help perf to check first.

Actions #2

Updated by Anoop Saldanha about 11 years ago

  • Assignee set to Anoop Saldanha

Sounds good.

Actions #3

Updated by Anoop Saldanha about 11 years ago

1. This optimization is ideal when ones uses single mode.
2. In full mode, a sig might belong to multiple sghs. So the patterns in a sig which can be considered to be mpm-checked, should be present in every sgh the sig is part of.

Actions #4

Updated by Anoop Saldanha about 11 years ago

Anoop Saldanha wrote:

1. This optimization is ideal when ones uses single mode.
2. In full mode, a sig might belong to multiple sghs. So the patterns in a sig which can be considered to be mpm-checked, should be present in every sgh the sig is part of.

Dwelling a bit over (2), we can probably do 2 things -
2.1 A sig is updated with patterns that is common to all the sghs it belongs to.
2.2 A sig is updated with patterns for every sgh it belongs to, but each sgh pattern's stored in a section of its own inside the sig. So when a sig is inspected, we use the sgh index to use the patterns specific to the sgh-sig combination. The issue here is how much data we might end up storing in a sig.

Actions #5

Updated by Victor Julien over 10 years ago

  • Priority changed from Normal to Low
Actions #6

Updated by Victor Julien about 10 years ago

  • Target version changed from 2.0rc2 to 3.0RC2
Actions #7

Updated by Victor Julien over 9 years ago

  • Target version changed from 3.0RC2 to 70
Actions #8

Updated by Victor Julien over 8 years ago

  • Status changed from New to Closed
  • Assignee deleted (Anoop Saldanha)
  • Target version deleted (70)

This no longer makes sense. In the new mpm setup we use sid's instead of pid's.

Actions

Also available in: Atom PDF