Bug #5197: fast_pattern assignment of specific content results in FN - Suricata - Open Information Security Foundation

Actions

#1

Updated by Brandon Murphy about 2 years ago

For testing purposes, I created a variation of the rule, each with a different fast_pattern manually assigned, and one without a manual assignment.

alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:1;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:2;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; fast_pattern; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:4;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; fast_pattern; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:5;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;)
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; fast_pattern; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:7;)

here are the results.

03/17/2022-10:55:24.304304  [**] [1:1:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:4:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:5:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:6:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-10:55:24.304304  [**] [1:7:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429

Interesting to me that sid:3, and sid:6 actually have the same fast_pattern, just in different positions.

-------------------------------------------------------------------
Date: 19/3/2022 -- 15:44:40
-------------------------------------------------------------------
== Sid: 3 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; fast_pattern; content:"456789"; distance:2; within:6; sid:3;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: yes
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: 1
        Final content: 1

== Sid: 6 ==
alert dns any any -> $HOME_NET any (content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; fast_pattern; content:"|00 00 10 00 01|"; distance:0;  content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:6;)
    Fast Pattern analysis:
        Fast pattern matcher: content
        Flags: Within Distance
        Fast pattern set: yes
        Fast pattern only set: no
        Fast pattern chop set: no
        Original content: 1
        Final content: 1

============
Summary:
============
packet/stream payload, smallest pattern 1 byte(s), longest pattern 1 byte(s), number of patterns 2, avg pattern len 1.00 byte(s)

Actions

Copy link

#2

Updated by Victor Julien about 2 years ago

Status changed from New to Assigned
Assignee changed from OISF Dev to Victor Julien
Target version changed from TBD to 7.0.0-beta1

Actions

Copy link

#3

Updated by Shivani Bhardwaj about 2 years ago

s-v test: https://github.com/OISF/suricata-verify/pull/787

Actions

Copy link

#4

Updated by Victor Julien about 2 years ago

Assignee changed from Victor Julien to Jeff Lucovsky
Priority changed from Normal to High

I investigated this issue through the rules for #5162, but I think this is the correct ticket. It looks like when I comment out the contents of DetectContentPropagateLimits, it works. This suggests that the offset/depth calculated based on the chain of patterns is off somehow. Jeff, can you check if that is the case for you as well?

Actions

Copy link

#5

Updated by Jeff Lucovsky about 2 years ago

I'm calculating content length of 37. Still looking

Actions

Copy link

#6

Updated by Jeff Lucovsky about 2 years ago

Using the attached pcap, both of the rules (the first 2 posted), fire for me:

jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:232) <Warning> (ProcessSigFiles) -- [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /home/jlucovsky/5197.rules
[587417] 9/4/2022 -- 09:55:57 - (detect-engine-loader.c:347) <Warning> (SigLoadSignatures) -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1 rule files specified, but no rules were loaded!
[587417] 9/4/2022 -- 09:55:57 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1   Engine started.
[587417] 9/4/2022 -- 09:55:57 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[587431] 9/4/2022 -- 09:55:57 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ sudo rm -rf /tmp/ll/*;src/suricata -c suricata.yaml -S ~/rules//5197.rules -r ~/pcap/redacted_dns.pcapng -l /tmp/ll
[587476] 9/4/2022 -- 09:56:08 - (suricata.c:1141) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[587476] 9/4/2022 -- 09:56:09 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 16 FM: 1 FR: 1   Engine started.
[587476] 9/4/2022 -- 09:56:09 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[587490] 9/4/2022 -- 09:56:09 - (source-pcap-file.c:384) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
jlucovsky@ ~/src/jal/suricata (master) $ cat /tmp/ll/fast.log
03/17/2022-06:55:24.304304  [**] [1:8:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
03/17/2022-06:55:24.304304  [**] [1:9:0] (null) [**] [Classification: (null)] [Priority: 3] {UDP} 1.1.1.1:53 -> 10.127.0.9:56429
jlucovsky@ ~/src/jal/suricata (master) $ cat ~/rules/5197.rules
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; sid:8;)
alert dns any any -> $HOME_NET any (byte_test:1,&,128,3; content:"|00 01 00 01 00 00 00 00|"; offset:4; depth:8; content:"1"; distance:1; within:1; content:"|00 00 10 00 01|"; distance:0; content:"|00 10 00 01 00 00 00 01|"; distance:2; within:8; content:"1"; distance:3; within:1; content:"456789"; distance:2; within:6; fast_pattern; sid:9;)

Actions

Copy link Download all files

#7

Updated by Brandon Murphy about 2 years ago

File f92840348b24bc49_eve.json f92840348b24bc49_eve.json added
File 7139323980e636ee_suricata.yaml 7139323980e636ee_suricata.yaml added

I just tested again, just to make sure and was able to replicate only sid:1 alerting.

Attached are the config, and eve log used.

[2565] 10/4/2022 -- 02:36:36 - (suricata.c:1142) <Notice> (LogVersion) -- This is Suricata version 7.0.0-dev (8ef066318 2022-03-29) running in USER mode
[2565] 10/4/2022 -- 02:36:36 - (util-cpu.c:178) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 4
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- fast output device (regular) initialized: dalton-fast.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- eve-log output device (regular) initialized: dalton-eve.json
[2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[2565] 10/4/2022 -- 02:36:36 - (output-json-dnp3.c:288) <Info> (OutputDNP3LogInitSub) -- DNP3 log sub-module initialized.
[2565] 10/4/2022 -- 02:36:36 - (runmodes.c:658) <Warning> (RunModeInitializeEveOutput) -- [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - eve module 'ikev2' has been replaced by 'ike'
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- http-log output device (regular) initialized: dalton-http.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- tls-log output device (regular) initialized: dalton-tls.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- alert-debug output device (regular) initialized: dalton-alert_debug.log
[2565] 10/4/2022 -- 02:36:36 - (util-logopenfile.c:594) <Info> (SCConfLogOpenGeneric) -- stats output device (regular) initialized: dalton-stats.log
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-loader.c:355) <Info> (SigLoadSignatures) -- 1 rule files processed. 2 rules successfully loaded, 0 rules failed
[2565] 10/4/2022 -- 02:36:36 - (util-threshold-config.c:1113) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 0 rule(s) found
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1471) <Info> (SigAddressPrepareStage1) -- 2 signatures processed. 0 are IP-only rules, 2 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
[2565] 10/4/2022 -- 02:36:36 - (tm-threads.c:2040) <Notice> (TmThreadWaitOnThreadInit) -- Threads created -> RX: 1 W: 4 FM: 1 FR: 1   Engine started.
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:173) <Info> (ReceivePcapFileLoop) -- Starting file run for /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file-helper.c:157) <Info> (PcapFileDispatch) -- pcap file /tmp/f92840348b24bc49_Apr-10-2022_02-36-36/pcaps/redacted_dns.pcapng.pcap end of file reached (pcap err code 0)
[2565] 10/4/2022 -- 02:36:36 - (suricata.c:2756) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[2565] 10/4/2022 -- 02:36:36 - (suricata.c:1161) <Info> (SCPrintElapsedTime) -- time elapsed 0.080s
[2566] 10/4/2022 -- 02:36:36 - (source-pcap-file.c:389) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 2 packets, 225 bytes
[2567] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2568] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2569] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2570] 10/4/2022 -- 02:36:36 - (log-tlslog.c:200) <Info> (LogTlsLogExitPrintStats) -- TLS logger logged 0 requests
[2565] 10/4/2022 -- 02:36:36 - (counters.c:857) <Info> (StatsLogSummary) -- Alerts: 1
[2565] 10/4/2022 -- 02:36:36 - (detect-engine-build.c:1773) <Info> (SigAddressCleanupStage1) -- cleaning up signature grouping structure... complete

Actions

Copy link

#8

Updated by Jeff Lucovsky about 2 years ago

When Suricata is configured to not use hyperscan, I can reproduce the results (one, rather than two, alerts).

mpm-algo: ac
spm-algo: bm

Actions

Copy link

#9

Updated by Victor Julien over 1 year ago

Status changed from Assigned to Closed
Assignee changed from Jeff Lucovsky to Victor Julien
Priority changed from High to Normal

This has been fixed as part of #5162, so its essentially a duplicate.

Actions

Copy link

#10

Updated by Victor Julien over 1 year ago

Is duplicate of Bug #5162: inspection of smb traffic without smb/dcerpc doesn't work correct. added

redacted_dns.pcapng (436 Bytes) redacted_dns.pcapng		Brandon Murphy, 03/19/2022 12:24 AM
f92840348b24bc49_eve.json (9.33 KB) f92840348b24bc49_eve.json		Brandon Murphy, 04/10/2022 02:38 AM
7139323980e636ee_suricata.yaml (70.1 KB) 7139323980e636ee_suricata.yaml		Brandon Murphy, 04/10/2022 02:38 AM

Project

General

Profile

Suricata

Custom queries

Bug #5197

fast_pattern assignment of specific content results in FN

Updated by Brandon Murphy about 2 years ago

Updated by Victor Julien about 2 years ago

Updated by Shivani Bhardwaj about 2 years ago

Updated by Victor Julien about 2 years ago

Updated by Jeff Lucovsky about 2 years ago

Updated by Jeff Lucovsky about 2 years ago

Updated by Brandon Murphy about 2 years ago

Updated by Jeff Lucovsky about 2 years ago

Updated by Victor Julien over 1 year ago

Updated by Victor Julien over 1 year ago