Bug #2917
closedUnable to find the sm in any of the sm lists
Added by Peter Manev about 5 years ago. Updated 10 months ago.
Description
How to reproduce - enable all rules in an ETPro ruleset and load them.
suricata -T -S "rules/*.rules" [36522] 3/4/2019 -- 04:02:13 - (suricata.c:1851) <Info> (ParseCommandLine) -- Running suricata under test mode [36522] 3/4/2019 -- 04:02:13 - (suricata.c:1064) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (rev 7f0bdc662) [36522] 3/4/2019 -- 04:03:04 - (detect-parse.c:576) <Error> (SigMatchListSMBelongsTo) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Unable to find the sm in any of the sm lists [36522] 3/4/2019 -- 04:03:04 - (suricata.c:2971) <Notice> (main) -- Configuration provided was successfully loaded. Exiting.
Updated by Andreas Herz almost 5 years ago
- Assignee set to OISF Dev
- Target version set to TBD
Updated by Victor Julien over 4 years ago
I'm not seeing this with current rules. @Peter Manev do you still have the set to reproduce this?
Updated by Peter Manev over 4 years ago
I dont see it anymore with the 5.0 ruleset.
Updated by Peter Manev about 4 years ago
Tested with - Suricata version 6.0.0-dev (0c5c2173b 2020-03-14)
Thanks to Brandon M (PFPT) it is now possible to narrow down and reproduce the issue.
alert http $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"TEST 1"; flow:to_server,established; content:"GET"; http_method; flowbits:set,test; sid:1;) # does not work - based on the original rules alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/\s200\s/"; sid:2;) # works - removing the flowbit #alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; pcre:"/\s200\s/"; sid:2;) # works - sub pcre with content - this validates it something with the flowbit relationship? #alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; content:"|20|200|20|"; sid:2;) # works - add a buffer to the pcre #alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/200/S"; sid:2;)
It seems "Unable to find the sm in any of the sm lists" gets triggered specifically by a combination of flowbits plus non buffered pcre ...
The rules above are definitely not optimal - they just depict the issue and make it reproducible.
Updated by Jeff Barber over 3 years ago
An additional way of reproducing this in the released 6.0.0 version is this rule:
alert dns any any -> any any (msg:"VU#257161:CVE-2020-11901 DNS malformed response to provide incorrect size for heap allocation error"; isdataat:129 ; app-layer-event:dns.malformed_data; sid:1371257161; rev:1;)
That was my attempt to adapt a snort rule from https://github.com/CERTCC/PoC-Exploits/blob/master/vu-257161/vu-257161.rules, which uses a suricata-rejected pairing of app-layer-event and dsize (following the suggestion at https://suricata.readthedocs.io/en/suricata-6.0.0/rules/differences-from-snort.html#don-t-cross-the-streams).
alert dns any any -> any any (msg:"VU#257161:CVE-2020-11901 DNS malformed response to provide incorrect size for heap allocation error"; dsize:>128 ; app-layer-event:dns.malformed_data; sid:1371257161; rev:1;)
Updated by Shivani Bhardwaj about 1 year ago
I cannot reproduce this on the latest master or master-6.0.x. However, I can reproduce this on the commit tagged by Peter for 6.0.x 0c5c2173bc1
Not sure which commit fixed this. Should I do a further analysis?
Updated by Victor Julien about 1 year ago
Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.
Updated by Shivani Bhardwaj about 1 year ago
Victor Julien wrote in #note-11:
Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.
I was able to pinpoint the fix to 227d1d967a5902 which by the message does not seem like intended to fix this issue. In fact, it seems to be suppressing error message for pcre only rules. Do you think the issue might still persist or would this commit be considered a fix for this certain issue?
Updated by Victor Julien about 1 year ago
- Status changed from New to In Progress
- Assignee changed from OISF Dev to Shivani Bhardwaj
Updated by Shivani Bhardwaj 10 months ago
- Target version changed from TBD to 7.0.1
Updated by Shivani Bhardwaj 10 months ago
- Status changed from In Progress to Closed
- Target version changed from 7.0.1 to 7.0.0
After some research, it was established that the control goes to AppendStreamInspectEngine/SigMatchListSMBelongsTo for stream inspection and match (for the list DETECT_SM_LIST_PMATCH) which happens for flowbits rules with read ops like isset, isnotset and all pcre rules.
But, if the stream was not mpm, the fn SigMatchListSMBelongsTo was throwing this error Unable to find the sm in any of the sm lists. However, this error message was shown even for cases where there was no SigMatch to match against. As this can happen in certain rules like the pcre ones mentioned here, this error was suppressed as a part of 227d1d967a5902.
As for the dependency on flowbits, we found some other issues for which we're making tickets and will add the relationship to this ticket for better tracking.
Thanks all for your input!
Updated by Shivani Bhardwaj 10 months ago
- Related to Bug #6177: detect-engine: stream match for rules is interdependent added
Updated by Shivani Bhardwaj 10 months ago
- Related to Bug #6178: dns: erroneous app_proto settings in rule analysis added