Suricata

can you pin point a specific rule?

I'm not seeing this with current rules. @Peter Manev do you still have the set to reproduce this?

I dont see it anymore with the 5.0 ruleset.

Tested with - Suricata version 6.0.0-dev (0c5c2173b 2020-03-14)

Thanks to Brandon M (PFPT) it is now possible to narrow down and reproduce the issue.

alert http $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"TEST 1"; flow:to_server,established; content:"GET"; http_method; flowbits:set,test; sid:1;)

# does not work - based on the original rules
alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/\s200\s/"; sid:2;)

# works - removing the flowbit
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; pcre:"/\s200\s/"; sid:2;)

# works - sub pcre with content - this validates it something with the flowbit relationship? 
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; content:"|20|200|20|"; sid:2;)

# works - add a buffer to the pcre
#alert http $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"TEST 2"; flowbits:isset,test; pcre:"/200/S"; sid:2;)

It seems "Unable to find the sm in any of the sm lists" gets triggered specifically by a combination of flowbits plus non buffered pcre ...
The rules above are definitely not optimal - they just depict the issue and make it reproducible.

An additional way of reproducing this in the released 6.0.0 version is this rule:

alert dns any any -> any any (msg:"VU#257161:CVE-2020-11901 DNS malformed response to provide incorrect size for heap allocation error"; isdataat:129 ; app-layer-event:dns.malformed_data; sid:1371257161; rev:1;)

That was my attempt to adapt a snort rule from https://github.com/CERTCC/PoC-Exploits/blob/master/vu-257161/vu-257161.rules, which uses a suricata-rejected pairing of app-layer-event and dsize (following the suggestion at https://suricata.readthedocs.io/en/suricata-6.0.0/rules/differences-from-snort.html#don-t-cross-the-streams).

alert dns any any -> any any (msg:"VU#257161:CVE-2020-11901 DNS malformed response to provide incorrect size for heap allocation error"; dsize:>128 ; app-layer-event:dns.malformed_data; sid:1371257161; rev:1;)

I cannot reproduce this on the latest master or master-6.0.x. However, I can reproduce this on the commit tagged by Peter for 6.0.x 0c5c2173bc1

Not sure which commit fixed this. Should I do a further analysis?

Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.

Victor Julien wrote in #note-11:

Yeah a git bisect would be good to see if there was a real fix or perhaps an accidental fix.

I was able to pinpoint the fix to 227d1d967a5902 which by the message does not seem like intended to fix this issue. In fact, it seems to be suppressing error message for pcre only rules. Do you think the issue might still persist or would this commit be considered a fix for this certain issue?