Suricata

Summary:
Suricata is not always alerting on traffic that matches content in rules.

Details:
When testing Suricata on my virtual machine (Virtual Box 5.2.6) running CentOS Linux release 7.4.1708 (Core), I found there were some situations when traffic was not triggering alerts but was still being picked up in the http.log. As best as I can tell, the behavior is influenced by the combination of the location of the content in the packets and also whether or not the content is split between packets , though it cannot reliably be reproduced. For my tests I was using nc (Ncat version 6.40 http://nmap.org/ncat) to serve the content on the VM where Suricata is hosted, and then cURL on a separate physical machine to retrieve it from the VM with Suricata. There is NAT'ing and a port forward to the VM running Suricata in this case.

It is possible there is something weird going on in the way the traffic is passed to the VM so that Suricata is behaving as expected, but I wanted to report this issue anyway especially after reading Bug #2427.

I was able to reproduce this on Suricata 3.x versions and also 4.x versions (4.0.3, 4.0.4, 4.1.0-beta1). It is worth noting that the 3.x versions had some differences in what traffic was picked up when compared to 4.x versions. I was also more reliably able to reproduce the behavior on the 3.x versions, meaning when content was in a specific location it always did NOT trigger an alert. For the sake of the issue here I plan to focus on the 4.x versions.

The rules file is attached as custom.rules. I tried to make my rules as simple as possible so that there was not much room for error.

I ran some packet captures on the VM where Suricata is located. These should allow for reliably reproducing the behavior I have seen. The attachment pcaps.tar.gz contains these captures. I tried to name the captures to describe the expected results. There is even one capture where the same payload is served three times yet only one of the three streams triggers an alert. Here are some details on the captures:

• 11114chars_catch-meContent_MISS.pcap

This capture does NOT trigger an alert in my tests but it seems should trigger the "catch-me" rule. Content is split between frames 19 and 20.

• 11121chars_catch-meContent_HIT.pcap

This capture does trigger an alert. The content "catch-me" is contained in frame 20.

• 5842chars_hitContent_1and2MISS_3ALERT.pcap

This capture has 3 streams and should match the rule with the "hit" content. The first 2 streams do NOT trigger alerts and the last one does. It is the same payload served in all three cases.

• 5842chars_hitContent_MISS.pcap

This capture does NOT trigger an alert. The "hit" content is split between frames 11 and 12.

Steps to reproduce:
1. Use vanilla suricata.yaml config. Set HOME_NET to appropriate address and update rule-files section to point to the attached custom.rules file. (Tested with both ac and hs mpm-algos and reproducible in both cases)
2. Run Suricata in pcap file mode with attached packet captures using "suricata -r <pcap name>"

Actual Behavior:
The traffic in the captures does not always trigger alerts even though the rules seem to be configured in a way that should match the traffic. Every stream is logged in the http.log file.

Expected Behavior:
The traffic in the captures should trigger for every stream since they each contain content that match in custom.rules.