Project

General

Profile

Actions

Bug #9285

open
SD

detect: out-of-order first data segment is matched by packet rules before app-layer detection

Bug #9285: detect: out-of-order first data segment is matched by packet rules before app-layer detection

Added by Shane Dugan 13 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Hi Suricata team,

In IPS mode, a generic "drop other TCP" rule is dropping legitimate TLS flows when the ClientHello is split across two segments, the first segment is lost, and the second one arrives first. Is this expected behavior, or is there an existing setting we've missed?

Description

If the first client data segment of a TCP flow is lost and a later segment arrives first, Suricata runs packet rules against that later segment before app-layer protocol detection has run. At that point the flow's protocol is still unknown. With a policy like "pass TLS, drop any other established to_server TCP", the drop rule matches and the segment is dropped. When the missing segment arrives and fills the gap, the flow is correctly detected as TLS, but the earlier segment has already been dropped.

This arrival order comes from normal TCP loss recovery, not unusual network reordering. Under RACK-TLP (RFC 8985), a client whose first flight goes unacknowledged retransmits the last outstanding segment first as a tail-loss probe, and only resends the first segment when the RTO fires. TLS ClientHellos are more and more often sent as two segments; for example, a post-quantum X25519MLKEM768 key share brings the ClientHello to about 1.5 KB. So losing the first flight of the handshake produces exactly this order.

Steps to reproduce

Command:

suricata -c suricata.yaml -S repro.rules -r ooo-clienthello.pcap \
  -k none --simulate-ips --runmode single -l out/

repro.rules:

pass tls any any -> any any (msg:"allow TLS"; flow:to_server,established; sid:1;)
drop tcp any any -> any any (msg:"drop other established to_server TCP"; flow:to_server,established; sid:2;)

suricata.yaml (all other settings are defaults):

%YAML 1.1
---
outputs:
  - eve-log:
      enabled: yes
      filename: eve.json
      types:
        - alert
        - drop:
            alerts: yes
            flows: all
        - tls
        - flow
app-layer:
  protocols:
    tls:
      enabled: yes
      detection-ports:
        dp: 443

ooo-clienthello.pcap contains a single synthetic flow, 192.0.2.10 -> 198.51.100.20:443:

# Direction Packet
1-3 3-way handshake (MSS 1460)
4 client -> server data at relative seq 1449, 2 bytes (tail of the ClientHello, arrives first)
5 server -> client duplicate ACK 1
6 client -> server data at relative seq 1, 1448 bytes (head of the ClientHello, fills the gap)
7+ ServerHello, FIN exchange

inorder-clienthello.pcap is the same flow with packets 4 and 6 swapped. It is the control.

Expected behavior

No drop. The flow is detected as TLS and passed by sid 1.

Actual behavior

Sid 2 drops packet 4. Its drop event has no app_proto, and the flow event later reports app_proto: tls. The in-order control produces no drops.

Version Out-of-order pcap In-order pcap
main (9.0.0-dev, 9451642) 1 drop (packet 4, sid 2) 0 drops
8.0.7 1 drop (packet 4, sid 2) 0 drops

We ran each case 3 times and got the same result every time. Both builds are unmodified upstream sources: the 8.0.7 release tarball and a GitHub main checkout.

Question

Is this expected? Is there a stream or detection setting that holds back packet-rule evaluation until the start of the stream is contiguous?

As a local experiment, we deferred packet-rule evaluation for a to_server packet when all of these hold:

  • it sits beyond a gap at the very start of the client stream;
  • no app-layer data has been consumed yet;
  • the gap is at most one segment;
  • the receiver hasn't ACKed past the stream base.

The deferred packet is inspected normally once the gap fills, or at flow timeout if the gap never fills. This fixes the case above, but we'd rather follow upstream's direction. Does this belong in stream reassembly, in protocol detection, or somewhere else?

Attachments


Files

ooo-clienthello.pcap (2.51 KB) ooo-clienthello.pcap Shane Dugan, 10/26/2026 06:01 PM
inorder-clienthello.pcap (2.51 KB) inorder-clienthello.pcap Shane Dugan, 10/26/2026 06:01 PM
suricata.yaml (352 Bytes) suricata.yaml Shane Dugan, 10/26/2026 06:01 PM
repro.rules (191 Bytes) repro.rules Shane Dugan, 10/26/2026 06:01 PM

No data to display

Actions

Also available in: PDF Atom