Suricata

Summary:

We encountered a situation where a single connection contains traffic that has multiple rule matches, yet there are no alerts being generated. This behavior seems to be affected by the part of a connection captured in the packet capture.

Details:

There was an email sent to OISF-users at https://lists.openinfosecfoundation.org/pipermail/oisf-users/2018-October/016297.html. That has full details of the situation and how it was discovered in our environment. The short of it is that we saw a large difference in the number of alerts between our sensor sets and these alerts were all produced from a single flow of SMB (specifically SMB2) traffic. In our environment we have two sensor sets that are configured to receive identical copies of traffic. The sets are identical in terms of hardware, OS, Suricata version (4.0.5), and Suricata configuration.

I was able to gather two ~500MB packet captures traffic on the sensor where alerts were missing. When one of these captures is run through Suricata in pcap file mode, then all expected alerts are generated. The other capture produces no alerts even though there is traffic that matches specific SMB rules. If I remove a single frame from the capture where no alerts are generated, then many alerts are generated. This single frame is an SMB Close Response message. If the capture is modified to start with any SMB Close Response message in the capture, then no alerts are generated even though there are matches through the capture.

Unfortunately, I am not able to provide either of these 500MB captures. The good news is that I was able to recreate a similar situation using publicly available SMB/SMB2 pcaps that have some modifications. The captures trigger alerts on different rules than the ones we observed in our environment, but are still rules that look for SMB traffic (though they are not SMB protocol rules). The captures were found at https://github.com/401trg/detections/tree/master/pcaps and I will attach these along with the modified files to this issue. The situation was reproducible using an almost vanilla configuration and also was reproducible on Suricata 3.1.

Attached you will find packet captures, the suricata.yaml config file I used to recreate this, and also the --build-info output.

Actual Behavior:

Running a packet capture through Suricata in pcap file mode does not produce any alerts. The captures contain traffic that match loaded rules. When removing a single frame from this capture, then rules are triggered.

Expected Behavior:

The traffic should generate alerts since there are rules being used that match traffic in the captures.

Steps to Reproduce:

On Suricata 4.0.5, use the attached suricata.yaml, custom.rules file, and pcap files.

Run the pcaps through Suricata in pcap file mode using:
suricata -c suricata.yaml -r <pcap file>

One example of the exact command I ran is:
sudo suricata -vv -c suricata.yaml -k none --pidfile pcap.pid -l logging -r pcaps/C_20171220_smb_mimikatz_copy_SMB2_frame_26_NOT_close.pcap

Here are details on the number of alerts triggered by each capture attached. There are 4 groups, A through D. Note that group A for some reason did not produce any alerts even though it does not start with an SMB Close Respones, so it appears this is not the only thing affecting this behavior. The file names contain the frame number the captures start relative to the original capture for each group.

alerts,pcap_name
3,A_20171220_smb_metasploit_psexec_pth_download_meterpreter.pcap
0,A_20171220_smb_metasploit_psexec_pth_download_meterpreter_frame_31_close.pcap
0,A_20171220_smb_metasploit_psexec_pth_download_meterpreter_frame_32_NOT_close.pcap

3,B_20171220_smb_mimikatz_copy_to_host.pcap
0,B_20171220_smb_mimikatz_copy_to_host_SMB2_frame_15_close.pcap
2,B_20171220_smb_mimikatz_copy_to_host_SMB2_frame_16_NOT_close.pcap

4,C_20171220_smb_mimikatz_copy.pcap
0,C_20171220_smb_mimikatz_copy_SMB2_frame_25_close.pcap
4,C_20171220_smb_mimikatz_copy_SMB2_frame_26_NOT_close.pcap

2,D_20171220_smb_psexec_mimikatz_ticket_dump.pcap
0,D_20171220_smb_psexec_mimikatz_ticket_dump_SMB2_frame_49_close.pcap
1,D_20171220_smb_psexec_mimikatz_ticket_dump_SMB2_frame_50_NOT_close.pcap

Only the rules with these SIDs from Emerging Threats are triggered for pcap each group:

group,sid
A,2025719
B,2025701
C,2025701
D,2025701