Project

General

Profile

Actions
Copy link

Bug #5197

closed

fast_pattern assignment of specific content results in FN

Added by Brandon Murphy about 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Victor Julien
Target version:
7.0.0-beta1
Affected Versions:
4.1.10, 5.0.8, 6.0.4, git master
Effort:
Difficulty:
Label:

Description

Consider the following two rules, designed to detect a specific answer for a DNS TXT record reply cough more dns buffers in suri7 plz! cough. The only difference is that sid:2 has a manual fast_pattern assignment.

alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;)
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)

In testing against the attached pcap, only sid:1; fires, despite containing the exact same signature logic.

This issue is not present in Suricata 4.0.x

Suricata shows the fast_patterns for each rule as follows

-------------------------------------------------------------------
Date: 19/3/2022 -- 00:14:39
-------------------------------------------------------------------
== Sid: 1 ==
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: no
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: \x00\x10\x00\x01\x00\x00\x00\x01
        Final content: \x00\x10\x00\x01\x00\x00\x00\x01

== Sid: 2 ==
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: yes
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: 456789
        Final content: 456789

============
Summary:
============
packet/stream payload, smallest pattern 6 byte(s), longest pattern 8 byte(s), number of patterns 2, avg pattern len 7.00 byte(s)

Files

Download all files
redacted_dns.pcapng (436 Bytes) redacted_dns.pcapng Brandon Murphy, 03/19/2022 12:24 AM
f92840348b24bc49_eve.json (9.33 KB) f92840348b24bc49_eve.json Brandon Murphy, 04/10/2022 02:38 AM
7139323980e636ee_suricata.yaml (70.1 KB) 7139323980e636ee_suricata.yaml Brandon Murphy, 04/10/2022 02:38 AM

Related issues 1 (0 open — 1 closed)

Is duplicate of Suricata - Bug #5162: inspection of smb traffic without smb/dcerpc doesn't work correct. ClosedVictor JulienActions
Actions
Copy link
 #1

Updated by Brandon Murphy about 2 years ago

For testing purposes, I created a variation of the rule, each with a different fast_pattern manually assigned, and one without a manual assignment.

alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; fast_pattern; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:4;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; fast_pattern; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:5;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; fast_pattern; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:7;)

here are the results.

03/17/2022-10:55:24.304304  [**] [1:1:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:4:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:5:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:6:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:7:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429

Interesting to me that sid:3, and sid:6 actually have the same fast_pattern, just in different positions.

-------------------------------------------------------------------
Date: 19/3/2022 -- 15:44:40
-------------------------------------------------------------------
== Sid: 3 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: yes
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: 1
        Final content: 1

== Sid: 6 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: yes
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: 1
        Final content: 1

============
Summary:
============
packet/stream payload, smallest pattern 1 byte(s), longest pattern 1 byte(s), number of patterns 2, avg pattern len 1.00 byte(s)
Actions
Copy link
 #2

Updated by Victor Julien about 2 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Victor Julien
  • Target version changed from TBD to 7.0.0-beta1
Actions
Copy link
 #4

Updated by Victor Julien about 2 years ago

  • Assignee changed from Victor Julien to Jeff Lucovsky
  • Priority changed from Normal to High

I investigated this issue through the rules for #5162, but I think this is the correct ticket. It looks like when I comment out the contents of DetectContentPropagateLimits, it works. This suggests that the offset/depth calculated based on the chain of patterns is off somehow. Jeff, can you check if that is the case for you as well?

Actions
Copy link
 #5

Updated by Jeff Lucovsky about 2 years ago

I'm calculating content length of 37. Still looking

Actions
Copy link
 #6

Updated by Jeff Lucovsky about 2 years ago

Using the attached pcap, both of the rules (the first 2 posted), fire for me:

jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:232) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /home/jlucovsky/5197.rules
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:347) <Warning> (SigLoadSignatures) -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1 rule files specified, but no rules were loaded!
[587417] 9/4/2022 -- 09:55:57 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1   Engine started.
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[587431] 9/4/2022 -- 09:55:57 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/rules//5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587476] 9/4/2022 -- 09:56:08 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587476] 9/4/2022 -- 09:56:09 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1   Engine started.
[587476] 9/4/2022 -- 09:56:09 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[587490] 9/4/2022 -- 09:56:09 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ cat /tmp/ll/fast.log
03/17/2022-06:55:24.304304  [**] [1:8:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-06:55:24.304304  [**] [1:9:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
jlucovsky@ ~/src/jal/suricata (master) $ cat ~/rules/5197.rules
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:8;)
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:9;)

Actions
Copy link Download all files
 #7

Updated by Brandon Murphy about 2 years ago

I just tested again, just to make sure and was able to replicate only sid:1 alerting.

Attached are the config, and eve log used.

[2565] 10/4/2022 -- 02:36:36 - (suricata.c:1142) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[2565] 10/4/2022 -- 02:36:36 - (util-cpu.c:178) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 4
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- fast output device (regular) initialized: dalton-fast.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- eve-log output device (regular) initialized: dalton-eve.json
[2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[2565] 10/4/2022 -- 02:36:36 - (runmodes.c:658) <Warning> (RunModeInitializeEveOutput) -- [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - eve module 'ikev2' has been replaced by 'ike'
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- http-log output device (regular) initialized: dalton-http.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- tls-log output device (regular) initialized: dalton-tls.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- alert-debug output device (regular) initialized: dalton-alert_debug.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- stats output device (regular) initialized: dalton-stats.log
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-loader.c:355) <Info> (SigLoadSignatures) -- 1 rule files processed. 2 rules successfully loaded, 0 rules failed
[2565] 10/4/2022 -- 02:36:36 - (util-threshold-config.c:1113) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 0 rule(s) found
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1471) <Info> (SigAddressPrepareStage1) -- 2 signatures processed. 0 are IP-only rules, 2 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
[2565] 10/4/2022 -- 02:36:36 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 4 FM: 1 FR: 1   Engine started.
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:173) <Info> (ReceivePcapFileLoop) -- Starting file run for /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file-helper.c:157) <Info> (PcapFileDispatch) -- pcap file /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap end of file reached (pcap err code 0)
[2565] 10/4/2022 -- 02:36:36 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[2565] 10/4/2022 -- 02:36:36 - (suricata.c:1161) <Info> (SCPrintElapsedTime) -- time elapsed 0.080s
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:389) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
[2567] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2568] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2569] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2570] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2565] 10/4/2022 -- 02:36:36 - (counters.c:857) <Info> (StatsLogSummary) -- Alerts: 1
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1773) <Info> (SigAddressCleanupStage1) -- cleaning up signature grouping structure... complete
Actions
Copy link
 #8

Updated by Jeff Lucovsky about 2 years ago

When Suricata is configured to not use hyperscan, I can reproduce the results (one, rather than two, alerts).

mpm-algo: ac
spm-algo: bm

Actions
Copy link
 #9

Updated by Victor Julien over 1 year ago

  • Status changed from Assigned to Closed
  • Assignee changed from Jeff Lucovsky to Victor Julien
  • Priority changed from High to Normal

This has been fixed as part of #5162, so its essentially a duplicate.

Actions
Copy link
 #10

Updated by Victor Julien over 1 year ago

  • Is duplicate of Bug #5162: inspection of smb traffic without smb/dcerpc doesn't work correct. added
Actions
Copy link

Also available in: Atom PDF