Suricata

Summary¶

response_body_identity_cl_known accepts a TCP gap inside a Content-Length body. When response_content_encoding_processing is gzip (or another compressed encoding), the gap arm calls response_body_data(None), which finishes the decompressor and does not put it back, then leaves response_body_data_left above zero. The next real body chunk hits response_decompressor.take().ok_or(ERROR). That stream error makes HTPHandleResponseData return a parser failure, and AppLayerParserParse calls StreamTcpDisableAppLayer on the flow. file_data never sees the plaintext that lives past the hole, and the next HTTP response on the same connection produces no transaction and no alert http match. Unpack suricata-http-ce-gap-applayer-off.tar.gz and run ./verify.sh.

Affected Versions¶

Confirmed on jasonish/suricata:latest (This is Suricata version 8.0.7 RELEASE) with the image's default yaml. HTTP/1 is enabled and registers APP_LAYER_PARSER_OPT_ACCEPT_GAPS. stream.reassembly.depth stays at the image default of 1 MiB. It was not set to 0, and HTTP and gap acceptance were not turned off. The same gap arm is present in the current tree's rust/htp/src/response.rs.

Preconditions¶

• The sensor is parsing the flow as HTTP/1. A request-side gap does not take this arm.
• The response has a compressed Content-Encoding (gzip, deflate, zlib, br, or lzma) and a known Content-Length, so the parser is in BodyIdentityCLKnown.
• A sequence hole sits in the middle of that body. Bytes of the same body are sent after the hole, and an ACK covers the sequence past the hole. A hole that swallows the entire remainder finalizes the body in the same call and does not reach the next-chunk error.
• The hole is far below the default 1 MiB reassembly depth. This run uses a 404-byte hole in a 6835-byte gzip body.

The endpoints have to present that ACK. A client that stops its ACK at the last contiguous byte leaves Suricata waiting, and this path does not run.

Security Impact¶

On the gapped connection, the file_data rule for the plaintext marker does not fire. EVE still has the first transaction (http.url /one, length 6431, which is 6835 minus the 404-byte hole) and then nothing else. app_layer.tx.http stays 1, app_layer.error.http.parser is 1, and tcp.reassembly_gap is 1. The following plain response is not an HTTP transaction and does not match an alert http file_data rule.

The same bytes with no hole produce both file_data hits and two HTTP transactions (/one length 6835, /two length 13). app_layer.tx.http is 2 and both the gap counter and the parser-error counter stay 0.

The process keeps running. Other flows are not affected. A gap counter by itself is not the bug. The bug is the missed plaintext inspection plus the loss of every later HTTP transaction on that connection. Default app-layer.error-policy is ignore, so this IDS run does not drop packets. It stops parsing HTTP on the flow.

Vulnerable Code¶

rust/htp/src/response.rs, response_body_identity_cl_known. A compressed gap ends the decompressor and then still waits for the rest of the body:

if data.is_gap() {
// ...
if resp.response_content_encoding_processing == HtpContentEncoding::None {
// gap is forwarded to the body hook
} else {
// end decompression on gap
self.response_body_data(None)?;
}
}
self.response_data_consume(data, bytes_to_consume as usize);
self.response_body_data_left = Some(left - bytes_to_consume);
if self.response_body_data_left > Some(0) {
return Err(HtpStatus::DATA);
}

response_body_data(None) takes the decompressor and does not put it back (response.rs around line 1079 and 1101). The None chunk adds nothing to response_message_len, which is why the logged length drops by the hole size. The next chunk is Some, so this guard does not save it:

if resp.response_decompressor.is_none() && data.is_none() {
return Ok(());
}
let mut decompressor = resp.response_decompressor.take().ok_or(HtpStatus::ERROR)?;

response_data turns that Err into HtpStreamState::ERROR (around line 1610). HTPHandleResponseData maps HTP_STREAM_STATE_ERROR to APP_LAYER_ERROR (src/app-layer-htp.c around lines 885 and 951). AppLayerParserParse then calls StreamTcpDisableAppLayer (src/app-layer-parser.c around lines 1381 and 1483).

request_body_identity feeds a request gap to the body hook and does not finish a decompressor. That is a different path.

Call Chain¶

1. ReassembleUpdateAppLayer delivers STREAM_GAP, then the post-gap bytes, once the ACK covers both (src/stream-tcp-reassemble.c).
2. HTPHandleResponseData calls htp_connp_response_data (src/app-layer-htp.c:883).
3. response_body_identity_cl_known sees data.is_gap() with a live content encoding and calls response_body_data(None) (rust/htp/src/response.rs:321), then returns DATA because response_body_data_left is still positive (response.rs:332).
4. The next body chunk calls response_body_data(Some) and take().ok_or(HtpStatus::ERROR) (response.rs:1079).
5. response_data sets HtpStreamState::ERROR (response.rs:1610).
6. HTPHandleResponseData returns APP_LAYER_ERROR (src/app-layer-htp.c:951).
7. AppLayerParserParse goes to error and calls StreamTcpDisableAppLayer (src/app-layer-parser.c:1381, src/app-layer-parser.c:1483).

Payload¶

One TCP connection, two HTTP/1.1 responses. The first is 200, Content-Type: text/plain, Content-Encoding: gzip, Content-Length: 6835, Connection: keep-alive. The gzip member is 6835 bytes. Its plaintext is 6000 incompressible bytes, the marker E2EGAPMARKER, then 800 more bytes. The attack sends the HTTP headers plus the first 6026 gzip bytes, skips 404 sequence numbers, then sends the remaining 405 gzip bytes. The client ACK moves past that hole before the second request. The second response is uncompressed 200 with body SECONDPLAINOK and Content-Length: 13.

Rules:

alert http any any -> any any (msg:"gzip plaintext marker"; flow:to_client,established; file_data; content:"E2EGAPMARKER"; sid:1000001; rev:1;)
alert http any any -> any any (msg:"second response body"; flow:to_client,established; file_data; content:"SECONDPLAINOK"; sid:1000002; rev:1;)

The control pcap is the same conversation with those 404 bytes present and the sequence numbers contiguous.

Reproduction¶

IDS pcap replay from jasonish/suricata:latest on the image default yaml. The container uses host networking. Checksums in the pcaps are valid, so checksum validation stays on. No --set overrides stream.reassembly.depth.

Two attachments.

suricata-http-ce-gap-applayer-off.tar.gz is this case. Unpack it and run ./verify.sh. It replays the control pcap, then the attack pcap, and compares EVE. ./stop_env.sh removes the container afterwards. A successful run ends like this:

-- control.json --
gzip_len=6835 prefix_len=6026 gap_len=0 suffix_len=0 packets=12
marker_alerts=1
second_alerts=1
http_urls=['/one', '/two']
tcp_reassembly_gap=0
http_parser_error=0
http_tx=2
http=[{'url': '/one', 'status': 200, 'length': 6835, 'http_method': 'GET'}, {'url': '/two', 'status': 200, 'length': 13, 'http_method': 'GET'}]
-- attack.json --
gzip_len=6835 prefix_len=6026 gap_len=404 suffix_len=405 packets=14
marker_alerts=0
second_alerts=0
http_urls=['/one']
tcp_reassembly_gap=1
http_parser_error=1
http_tx=1
http=[{'url': '/one', 'status': 200, 'length': 6431, 'http_method': 'GET'}]
RESULT: mid-body gzip hole drops the plaintext file_data hit and the next HTTP transaction; the same bytes without a hole do both
VERDICT: PASS

verify-suricata-http-ce-gap-applayer-off.tar.gz is a suricata-verify directory named suricata-http-ce-gap-applayer-off (the archive sits under sv/). Copy the unpacked directory into the suricata-verify tests/ tree and, from a built source tree, run:

python3 ../path/to/suricata-verify/run.py suricata-http-ce-gap-applayer-off

input.pcap is the gapped capture. test.rules is the two file_data rules above. The checks expect one alert for sid 1000001, one alert for sid 1000002, and an HTTP transaction for /two with status 200, plus the /one transaction. On 8.0.7 those alert counts are 0 and /two is absent, so run.py fails. After the fix the four checks should pass. python3 writepcap.py rebuilds input.pcap with the standard library only.

Fix Recommendation¶

A deflate stream cannot be resumed across missing bytes, so stopping decompression at the gap is reasonable. Turning that into HtpStreamState::ERROR is not.

In the compressed arm of response_body_identity_cl_known, after response_body_data(None), remember that the decompressor was closed because of a gap. Set response_content_encoding_processing to None or keep an explicit "decompression already finished" flag. Later body bytes must be counted toward Content-Length and must not call take().ok_or(HtpStatus::ERROR). Do not feed those still-compressed bytes to file_data. When response_body_data_left reaches zero, finalize the transaction. The full-remainder gap already does that, because the second response_body_data(None) hits the decompressor.is_none() && data.is_none() early return. The partial gap should finish the same way.

The connection parser then has to accept the next response. StreamTcpDisableAppLayer should not run for this hole. request_body_identity already survives a gap without destroying a decompressor. The response path should not be fatal either.