Suricata

A rule using fileext or file.name misses a file when an earlier file of the same transaction did not match, if the two files are inspected in different passes. With both files in one packet the same rule alerts.

Reproducer¶

One multipart POST to /up carrying a.txt and then b.bin.

alert http any any -> any any (msg:"fileext"; flow:established,to_server; http.uri; content:"/up"; fast_pattern; fileext:"bin"; sid:2;)
alert http any any -> any any (msg:"file.name"; flow:established,to_server; http.uri; content:"/up"; fast_pattern; file.name; content:"b.bin"; sid:3;)
alert http any any -> any any (msg:"control: file.data"; flow:established,to_server; http.uri; content:"/up"; fast_pattern; file.data; content:"second"; sid:4;)
alert http any any -> any any (msg:"control: filesize"; flow:established,to_server; http.uri; content:"/up"; fast_pattern; filesize:6; sid:5;)

Results on main (d996d85dcf), single runmode:

pcap	alerts
headers, a.txt part and b.bin part each in their own segment, each acked and followed by a toserver ACK	4, 5
the same bytes in one segment	2, 3, 4, 5

All four rules should alert once in both. sids 2 and 3 miss b.bin when it arrives after a.txt has been inspected.

Without the explicit fast_pattern on http.uri the file name content becomes the fast pattern, the prefilter brings the rule back as a new candidate when b.bin's name shows up, and the rule alerts. That hides the problem for the simplest rules.

Cause¶

The file.name inspect engine returns DETECT_ENGINE_INSPECT_SIG_CANT_MATCH_FILES once it has looked at every file present and none matched (src/detect-filename.c:277). DetectRunTxInspectRule() turns that into DE_STATE_FLAG_SIG_CANT_MATCH plus the engine's bit, and stores the state.

When a new file is opened on the transaction, stored rules are reopened by this test (src/detect.c:2467):

if (have_new_file && (item->flags & DE_STATE_FLAG_FILE_INSPECT)) {

DE_STATE_FLAG_FILE_INSPECT is the bit of engine id DE_STATE_ID_FILE_INSPECT, which is only given to engines on the legacy "files" list (src/detect-engine.c:796, 809, 829): filesize, filemd5 and friends. The file.name buffer engine gets an ordinary id, so its rules stay "can't match" for the rest of the transaction.

file.magic returns the same code from the same kind of engine (src/detect-filemagic.c:340); I did not test it. file.data is not affected in this test.

Impact¶

False negatives that depend on packetization, for protocols that carry several files in one transaction (HTTP multipart, SMTP).

Generator¶

from scapy.all import IP, TCP, Raw, wrpcap
B = b"xyzBOUNDARYxyz" 
def part(n, d):
    return (b"--" + B + b"\r\nContent-Disposition: form-data; name=\"f\"; filename=\"" + n +
            b"\"\r\nContent-Type: application/octet-stream\r\n\r\n" + d + b"\r\n")
p1 = part(b"a.txt", b"first"); p2 = part(b"b.bin", b"second") + b"--" + B + b"--\r\n" 
hdr = (b"POST /up HTTP/1.1\r\nHost: example.com\r\nContent-Type: multipart/form-data; boundary=" + B +
       b"\r\nContent-Length: " + str(len(p1) + len(p2)).encode() + b"\r\n\r\n")
resp = b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nOK" 
def pcap(name, segs, sport):
    S, D, P = "192.168.1.100", "192.168.1.1", 80; c, s = 100, 200
    pk = [IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="S",seq=c), IP(src=D,dst=S)/TCP(sport=P,dport=sport,flags="SA",seq=s,ack=c+1)]
    c += 1; s += 1; pk.append(IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="A",seq=c,ack=s))
    for seg in segs:
        pk.append(IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="PA",seq=c,ack=s)/Raw(load=seg)); c += len(seg)
        pk.append(IP(src=D,dst=S)/TCP(sport=P,dport=sport,flags="A",seq=s,ack=c))
        pk.append(IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="A",seq=c,ack=s))
    pk.append(IP(src=D,dst=S)/TCP(sport=P,dport=sport,flags="PA",seq=s,ack=c)/Raw(load=resp)); s += len(resp)
    pk.append(IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="FA",seq=c,ack=s))
    pk.append(IP(src=D,dst=S)/TCP(sport=P,dport=sport,flags="FA",seq=s,ack=c+1))
    pk.append(IP(src=S,dst=D)/TCP(sport=sport,dport=P,flags="A",seq=c+1,ack=s+1))
    wrpcap(name, pk)
pcap("split.pcap", [hdr, p1, p2], 40002)
pcap("joined.pcap", [hdr + p1 + p2], 40003)

The same code is in main-8.0.x (not run there).