Project

General

Profile

Actions

Feature #4321

open

Task #4380: tracking: improvements to bits, ints, vars

http2: Support link between packets in the same stream

Added by Jungho Yoon almost 4 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

There are multiple streams in http2. Since existing flowbits are based on the same flow, it is difficult to detect precisely in http2, which operates in a stream unit. For example, it is necessary to process only in the same stream2, but flowbits also handles the connection of stream1 and stream2 It's possible.

Existing flowbits may be improved, or similar options that can be processed on a per-stream basis are needed.

flowbits improvement or Examples:

streambits: set, name
streambits: isset, name
streambits: toggle, name
streambits: unset, name
streambits: isnotset, name
streambits: noalert

alert http2 any any -> any any (msg:"foo - PNG set"; flow:established,to_server; http2.header; content:"path /foo.html"; streambits:set,foo_png; )
alert http2 any any -> any any (msg:"foo - PNG isset"; flow:established,to_client; streambits:isset,foo_png; http2.frametype; content:"DATA"; pkt_data; content:"PNG"; )


Files

http2_streambtis.PNG (16.1 KB) http2_streambtis.PNG Jungho Yoon, 02/09/2021 08:06 AM
http.pcapng (11.7 KB) http.pcapng Jungho Yoon, 12/22/2021 06:16 AM
http2_multiplexing_1.pcapng (13.1 KB) http2_multiplexing_1.pcapng Jungho Yoon, 12/22/2021 06:16 AM
http2_multiplexing_2.pcapng (12.6 KB) http2_multiplexing_2.pcapng Jungho Yoon, 12/22/2021 06:16 AM

Related issues 2 (2 open0 closed)

Related to Suricata - Task #5488: Suricon 2022 brainstormAssignedVictor JulienActions
Related to Suricata - Feature #5665: rules: bidirectional transaction matchingIn ReviewPhilippe AntoineActions
Actions #1

Updated by Victor Julien over 3 years ago

  • Parent task set to #4380

I think this could be implemented using a more generic "transaction bits" in Suricata (txbits?) as we map streams to transactions in HTTP/2

Actions #2

Updated by Jungho Yoon over 3 years ago

txbits seems to be a clearer and better way.

Actions #3

Updated by Philippe Antoine over 3 years ago

I think it is possible to do it with a single signature as a HTTP2 stream is represented by a single Suricata transaction

Actions #4

Updated by Jungho Yoon over 3 years ago

Philippe Antoine wrote in #note-3:

I think it is possible to do it with a single signature as a HTTP2 stream is represented by a single Suricata transaction

I didn't understand. How is a single signature implemented?

In the situation where stream-1, stream-2, stream-3, etc. are mixed
Link (line flowbits) between stream-2 packets with different directions (to_server/to_client) or between the header of stream-2 and the pattern (ex. <xml> in compressed zip file) of a very large file of stream-2
Does this mean that the above situation is possible with a single signature?

Actions #5

Updated by Philippe Antoine about 3 years ago

Do you have a cap to test this ?
I would try with this single signature :
alert http2 any any -> any any (msg:"foo"; flow:established; http2.header; content:"path /foo.html"; file.data; content:"PNG"; )

Updated by Jungho Yoon almost 3 years ago

Test Environment

Suricata 6.0.4 read mode
 -  suricata -c /etc/suricata/suricata.yaml -k none --simulate-ips -r _PCAP_PATH_
Use default 6.0.4 suricata.yaml (change only http2 to enable)

pcap: http.pcapng, http2_multiplexing_1.pcapng
foo.txt (5,000 bytes)
- START1 ~ END1
bar.txt (5,000 bytes)
- START2 ~ END2

The client requests foo.txt and the server sends a file in response to the request.

In HTTP (http.pcapng), if each(to_server/to_client) directions of a flow match at once, rule writers can think of a single signature (sid:1) to match the URI and server response string at once.

alert http any any -> any any (msg:"http test 1"; flow:established; http.uri; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:1; )

However, sid:1 cannot be used because the flow direction is mixed. suricata also displays the message "mixes keywords with conflicting directions".

For this reason, rule writers like me use flowbits for detection.

alert http any any -> any any (msg:"http test 2"; flow:established; http.uri; content:"foo.txt"; flowbits:set,foo; sid:2; )
alert http any any -> any any (msg:"http test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:3; )

In HTTP, flowbits are not perfect for detecting a single transaction. HTTP can also cause some problems. "pipelining" and "single session multiple transactions".
Pipelining isn't a problem because it's seldom applied in practice.
In "single-session multiple-transaction", when using flowbits as in the situation above (sid:2/3), unintended transactions may be detected if the same string exists in other transactions.

However, by using flowbits:unset (eg, unset flowbits when HTTP method is detected), it is possible to distinguish whether a new HTTP transaction exists, so false positives can be prevented.
In my experience, I've yet to see any particularly problematic cases of using flowbits over HTTP.

But in HTTP2, by default, multiple transactions happen with multiplexing, so I thought a new option was needed.

First of all, like HTTP, I thought that other directions of flow would not be supported.

alert http2 any any -> any any (msg:"http2 test 1"; flow:established; http2.header; content:"foo.txt"; pkt_data; file.data; content:"END1"; sid:4; )

In suricata no "mixes keywords with conflicting directions" message and no alerts are generated for sid:4.
The foo.txt requested by the client exists in a path that can be seen as a URI. But I think the reason "mixes keywords with conflicting directions" didn't happen is because options like http2.header or file.data are used in both directions in HTTP2.
If HTTP2 is also processed within each(to_server/to_client) direction in a single transaction like HTTP, flowbits need to be used as in the above HTTP situation.

Rule writers will want detections to be processed in a single transaction.

alert http2 any any -> any any (msg:"http2 test 2"; flow:established; http2.header; content:"foo.txt"; flowbits:set,foo; sid:5; )
alert http2 any any -> any any (msg:"http2 test 3"; flow:established; flowbits:isset,foo; file.data; content:"END1"; sid:6; )

In http2_multiplexing_1.pcapng, sid:6 is a string match and is alerted. However, http2_multiplexing_2.pcapng also alerts. In this pcap, the string "END1" is contained in stream 15 (bar.txt) and "foo.txt" is contained in stream 13.

pcap: http2_multiplexing_2.pcapng
foo.txt (5,000 bytes)
- START1 ~ END2
bar.txt (5,000 bytes)
- START2 ~ END1

In this case, the alert on sid:6 is unintentional, as this is not the transaction we want to detect. The intended detection is foo.txt with the string "START1 to END1". However, the alert for sid:6 is raised because flowbits simply associates the presence of a string in the session.

Due to this problem, I think that a new option that can link a single transaction (stream) is needed. Of course, flowbits can use the existing behavior in HTTP2 as it is, so I think that new options should exist individually.

If there's anything I've misunderstood, please explain.

Actions #7

Updated by Philippe Antoine about 2 years ago

  • Related to Task #5488: Suricon 2022 brainstorm added
Actions #8

Updated by Victor Julien about 2 years ago

  • Subject changed from http2 - Support link between packets in the same stream to http2: Support link between packets in the same stream
Actions #9

Updated by Philippe Antoine about 1 year ago

  • Related to Feature #5665: rules: bidirectional transaction matching added
Actions #10

Updated by Philippe Antoine 5 months ago

  • Assignee set to OISF Dev
  • Target version set to TBD

The PR for #5665 should solve this

Actions

Also available in: Atom PDF