Bug #2917
closed
Unable to find the sm in any of the sm lists
Added by Peter Manev over 4 years ago.
Updated 3 months ago.
Description
How to reproduce - enable all rules in an ETPro ruleset and load them.
suricata -T -S "rules/*.rules"
[36522] 3/4/2019 -- 04:02:13 - (suricata.c:1851) <Info> (ParseCommandLine) -- Running suricata under test mode
[36522] 3/4/2019 -- 04:02:13 - (suricata.c:1064) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (rev 7f0bdc662)
[36522] 3/4/2019 -- 04:03:04 - (detect-parse.c:576) <Error> (SigMatchListSMBelongsTo) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Unable to find the sm in any of the sm lists
[36522] 3/4/2019 -- 04:03:04 - (suricata.c:2971) <Notice> (main) -- Configuration provided was successfully loaded. Exiting.
Related issues
2 (2 open — 0 closed)
can you pin point a specific rule?
- Assignee set to OISF Dev
- Target version set to TBD
- Status changed from New to Feedback
I'm not seeing this with current rules. @Peter Manev do you still have the set to reproduce this?
I dont see it anymore with the 5.0 ruleset.
- Status changed from Feedback to Closed
Tested with - Suricata version 6.0.0-dev (0c5c2173b 2020-03-14)
Thanks to Brandon M (PFPT) it is now possible to narrow down and reproduce the issue.
alert http $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"TEST 1"; flow:to_server,established; content:"GET"; http_method; flowbits:set,test; sid:1;)
# does not work - based on the original rules
alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/\s200\s/"; sid:2;)
# works - removing the flowbit
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; pcre:"/\s200\s/"; sid:2;)
# works - sub pcre with content - this validates it something with the flowbit relationship?
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; content:"|20|200|20|"; sid:2;)
# works - add a buffer to the pcre
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/200/S"; sid:2;)
It seems "Unable to find the sm in any of the sm lists" gets triggered specifically by a combination of flowbits plus non buffered pcre ...
The rules above are definitely not optimal - they just depict the issue and make it reproducible.
- Status changed from Closed to New
I cannot reproduce this on the latest master or master-6.0.x. However, I can reproduce this on the commit tagged by Peter for 6.0.x 0c5c2173bc1
Not sure which commit fixed this. Should I do a further analysis?
Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.
Victor Julien wrote in #note-11:
Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.
I was able to pinpoint the fix to 227d1d967a5902 which by the message does not seem like intended to fix this issue. In fact, it seems to be suppressing error message for pcre only rules. Do you think the issue might still persist or would this commit be considered a fix for this certain issue?
- Status changed from New to In Progress
- Assignee changed from OISF Dev to Shivani Bhardwaj
- Target version changed from TBD to 7.0.1
- Status changed from In Progress to Closed
- Target version changed from 7.0.1 to 7.0.0
After some research, it was established that the control goes to AppendStreamInspectEngine/SigMatchListSMBelongsTo for stream inspection and match (for the list DETECT_SM_LIST_PMATCH) which happens for flowbits rules with read ops like isset, isnotset and all pcre rules.
But, if the stream was not mpm, the fn SigMatchListSMBelongsTo was throwing this error Unable to find the sm in any of the sm lists. However, this error message was shown even for cases where there was no SigMatch to match against. As this can happen in certain rules like the pcre ones mentioned here, this error was suppressed as a part of 227d1d967a5902.
As for the dependency on flowbits, we found some other issues for which we're making tickets and will add the relationship to this ticket for better tracking.
Thanks all for your input!
- Related to Bug #6177: detect-engine: stream match for rules is interdependent added
- Related to Bug #6178: dns: erroneous app_proto settings in rule analysis added
Also available in: Atom
PDF