Task #4380: tracking: improvements to bits, ints, vars
http2 - Support link between packets in the same stream
There are multiple streams in http2. Since existing flowbits are based on the same flow, it is difficult to detect precisely in http2, which operates in a stream unit. For example, it is necessary to process only in the same stream2, but flowbits also handles the connection of stream1 and stream2 It's possible.
Existing flowbits may be improved, or similar options that can be processed on a per-stream basis are needed.
flowbits improvement or Examples:
streambits: set, name
streambits: isset, name
streambits: toggle, name
streambits: unset, name
streambits: isnotset, name
alert http2 any any -> any any (msg:"foo - PNG set"; flow:established,to_server; http2.header; content:"path /foo.html"; streambits:set,foo_png; )
alert http2 any any -> any any (msg:"foo - PNG isset"; flow:established,to_client; streambits:isset,foo_png; http2.frametype; content:"DATA"; pkt_data; content:"PNG"; )
Updated by Jungho Yoon over 1 year ago
Philippe Antoine wrote in #note-3:
I think it is possible to do it with a single signature as a HTTP2 stream is represented by a single Suricata transaction
I didn't understand. How is a single signature implemented?
In the situation where stream-1, stream-2, stream-3, etc. are mixed
Link (line flowbits) between stream-2 packets with different directions (to_server/to_client) or between the header of stream-2 and the pattern (ex. <xml> in compressed zip file) of a very large file of stream-2
Does this mean that the above situation is possible with a single signature?
Updated by Jungho Yoon 9 months ago
- File http.pcapng http.pcapng added
- File http2_multiplexing_1.pcapng http2_multiplexing_1.pcapng added
- File http2_multiplexing_2.pcapng http2_multiplexing_2.pcapng added
Suricata 6.0.4 read mode - suricata -c /etc/suricata/suricata.yaml -k none --simulate-ips -r _PCAP_PATH_ Use default 6.0.4 suricata.yaml (change only http2 to enable)
pcap: http.pcapng, http2_multiplexing_1.pcapng
foo.txt (5,000 bytes)
- START1 ~ END1
bar.txt (5,000 bytes)
- START2 ~ END2
The client requests foo.txt and the server sends a file in response to the request.
In HTTP (http.pcapng), if each(to_server/to_client) directions of a flow match at once, rule writers can think of a single signature (sid:1) to match the URI and server response string at once.
alert http any any -> any any (msg:"http test 1"; flow:established; http.uri; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:1; )
However, sid:1 cannot be used because the flow direction is mixed. suricata also displays the message "mixes keywords with conflicting directions".
For this reason, rule writers like me use flowbits for detection.
alert http any any -> any any (msg:"http test 2"; flow:established; http.uri; content:"foo.txt"; flowbits:set,foo; sid:2; ) alert http any any -> any any (msg:"http test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:3; )
In HTTP, flowbits are not perfect for detecting a single transaction. HTTP can also cause some problems. "pipelining" and "single session multiple transactions".
Pipelining isn't a problem because it's seldom applied in practice.
In "single-session multiple-transaction", when using flowbits as in the situation above (sid:2/3), unintended transactions may be detected if the same string exists in other transactions.
However, by using flowbits:unset (eg, unset flowbits when HTTP method is detected), it is possible to distinguish whether a new HTTP transaction exists, so false positives can be prevented.
In my experience, I've yet to see any particularly problematic cases of using flowbits over HTTP.
But in HTTP2, by default, multiple transactions happen with multiplexing, so I thought a new option was needed.
First of all, like HTTP, I thought that other directions of flow would not be supported.
alert http2 any any -> any any (msg:"http2 test 1"; flow:established; http2.header; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:4; )
In suricata no "mixes keywords with conflicting directions" message and no alerts are generated for sid:4.
The foo.txt requested by the client exists in a path that can be seen as a URI. But I think the reason "mixes keywords with conflicting directions" didn't happen is because options like http2.header or file.data are used in both directions in HTTP2.
If HTTP2 is also processed within each(to_server/to_client) direction in a single transaction like HTTP, flowbits need to be used as in the above HTTP situation.
Rule writers will want detections to be processed in a single transaction.
alert http2 any any -> any any (msg:"http2 test 2"; flow:established; http2.header; content:"foo.txt"; flowbits:set,foo; sid:5; ) alert http2 any any -> any any (msg:"http2 test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:6; )
In http2_multiplexing_1.pcapng, sid:6 is a string match and is alerted. However, http2_multiplexing_2.pcapng also alerts. In this pcap, the string "END1" is contained in stream 15 (bar.txt) and "foo.txt" is contained in stream 13.
foo.txt (5,000 bytes)
- START1 ~ END2
bar.txt (5,000 bytes)
- START2 ~ END1
In this case, the alert on sid:6 is unintentional, as this is not the transaction we want to detect. The intended detection is foo.txt with the string "START1 to END1". However, the alert for sid:6 is raised because flowbits simply associates the presence of a string in the session.
Due to this problem, I think that a new option that can link a single transaction (stream) is needed. Of course, flowbits can use the existing behavior in HTTP2 as it is, so I think that new options should exist individually.
If there's anything I've misunderstood, please explain.