Feature #4321
openTask #4380: tracking: improvements to bits, ints, vars
http2: Support link between packets in the same stream
Description
There are multiple streams in http2. Since existing flowbits are based on the same flow, it is difficult to detect precisely in http2, which operates in a stream unit. For example, it is necessary to process only in the same stream2, but flowbits also handles the connection of stream1 and stream2 It's possible.
Existing flowbits may be improved, or similar options that can be processed on a per-stream basis are needed.
flowbits improvement or Examples:
streambits: set, name
streambits: isset, name
streambits: toggle, name
streambits: unset, name
streambits: isnotset, name
streambits: noalert
alert http2 any any -> any any (msg:"foo - PNG set"; flow:established,to_server; http2.header; content:"path /foo.html"; streambits:set,foo_png; )
alert http2 any any -> any any (msg:"foo - PNG isset"; flow:established,to_client; streambits:isset,foo_png; http2.frametype; content:"DATA"; pkt_data; content:"PNG"; )
Files
Updated by Victor Julien over 3 years ago
- Parent task set to #4380
I think this could be implemented using a more generic "transaction bits" in Suricata (txbits?) as we map streams to transactions in HTTP/2
Updated by Jungho Yoon over 3 years ago
txbits seems to be a clearer and better way.
Updated by Philippe Antoine over 3 years ago
I think it is possible to do it with a single signature as a HTTP2 stream is represented by a single Suricata transaction
Updated by Jungho Yoon over 3 years ago
Philippe Antoine wrote in #note-3:
I think it is possible to do it with a single signature as a HTTP2 stream is represented by a single Suricata transaction
I didn't understand. How is a single signature implemented?
In the situation where stream-1, stream-2, stream-3, etc. are mixed
Link (line flowbits) between stream-2 packets with different directions (to_server/to_client) or between the header of stream-2 and the pattern (ex. <xml> in compressed zip file) of a very large file of stream-2
Does this mean that the above situation is possible with a single signature?
Updated by Philippe Antoine about 3 years ago
Do you have a cap to test this ?
I would try with this single signature :alert http2 any any -> any any (msg:"foo"; flow:established; http2.header; content:"path /foo.html"; file.data; content:"PNG"; )
Updated by Jungho Yoon almost 3 years ago
- File http.pcapng http.pcapng added
- File http2_multiplexing_1.pcapng http2_multiplexing_1.pcapng added
- File http2_multiplexing_2.pcapng http2_multiplexing_2.pcapng added
Test Environment
Suricata 6.0.4 read mode
- suricata -c /etc/suricata/suricata.yaml -k none --simulate-ips -r _PCAP_PATH_
Use default 6.0.4 suricata.yaml (change only http2 to enable)
pcap: http.pcapng, http2_multiplexing_1.pcapng
foo.txt (5,000 bytes)
- START1 ~ END1
bar.txt (5,000 bytes)
- START2 ~ END2
The client requests foo.txt and the server sends a file in response to the request.
In HTTP (http.pcapng), if each(to_server/to_client) directions of a flow match at once, rule writers can think of a single signature (sid:1) to match the URI and server response string at once.
alert http any any -> any any (msg:"http test 1"; flow:established; http.uri; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:1; )
However, sid:1 cannot be used because the flow direction is mixed. suricata also displays the message "mixes keywords with conflicting directions".
For this reason, rule writers like me use flowbits for detection.
alert http any any -> any any (msg:"http test 2"; flow:established; http.uri; content:"foo.txt"; flowbits:set,foo; sid:2; ) alert http any any -> any any (msg:"http test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:3; )
In HTTP, flowbits are not perfect for detecting a single transaction. HTTP can also cause some problems. "pipelining" and "single session multiple transactions".
Pipelining isn't a problem because it's seldom applied in practice.
In "single-session multiple-transaction", when using flowbits as in the situation above (sid:2/3), unintended transactions may be detected if the same string exists in other transactions.
However, by using flowbits:unset (eg, unset flowbits when HTTP method is detected), it is possible to distinguish whether a new HTTP transaction exists, so false positives can be prevented.
In my experience, I've yet to see any particularly problematic cases of using flowbits over HTTP.
But in HTTP2, by default, multiple transactions happen with multiplexing, so I thought a new option was needed.
First of all, like HTTP, I thought that other directions of flow would not be supported.
alert http2 any any -> any any (msg:"http2 test 1"; flow:established; http2.header; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:4; )
In suricata no "mixes keywords with conflicting directions" message and no alerts are generated for sid:4.
The foo.txt requested by the client exists in a path that can be seen as a URI. But I think the reason "mixes keywords with conflicting directions" didn't happen is because options like http2.header or file.data are used in both directions in HTTP2.
If HTTP2 is also processed within each(to_server/to_client) direction in a single transaction like HTTP, flowbits need to be used as in the above HTTP situation.
Rule writers will want detections to be processed in a single transaction.
alert http2 any any -> any any (msg:"http2 test 2"; flow:established; http2.header; content:"foo.txt"; flowbits:set,foo; sid:5; ) alert http2 any any -> any any (msg:"http2 test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:6; )
In http2_multiplexing_1.pcapng, sid:6 is a string match and is alerted. However, http2_multiplexing_2.pcapng also alerts. In this pcap, the string "END1" is contained in stream 15 (bar.txt) and "foo.txt" is contained in stream 13.
pcap: http2_multiplexing_2.pcapng
foo.txt (5,000 bytes)
- START1 ~ END2
bar.txt (5,000 bytes)
- START2 ~ END1
In this case, the alert on sid:6 is unintentional, as this is not the transaction we want to detect. The intended detection is foo.txt with the string "START1 to END1". However, the alert for sid:6 is raised because flowbits simply associates the presence of a string in the session.
Due to this problem, I think that a new option that can link a single transaction (stream) is needed. Of course, flowbits can use the existing behavior in HTTP2 as it is, so I think that new options should exist individually.
If there's anything I've misunderstood, please explain.
Updated by Philippe Antoine about 2 years ago
- Related to Task #5488: Suricon 2022 brainstorm added
Updated by Victor Julien about 2 years ago
- Subject changed from http2 - Support link between packets in the same stream to http2: Support link between packets in the same stream
Updated by Philippe Antoine about 1 year ago
- Related to Feature #5665: rules: bidirectional transaction matching added
Updated by Philippe Antoine 5 months ago
- Assignee set to OISF Dev
- Target version set to TBD
The PR for #5665 should solve this