Optimization #9173 » REPORT.md

Jason Ish, 10/06/2026 04:43 PM

Suricata: a multi-value `Content-Encoding` header silently disables HTTP body inspection

Reported by: Zer0Gate
Date: 2026-09-24
Component: rust/htp (libhtp-rs), HTTP/1 body decompression
Verified on: Suricata 8.0.7 RELEASE; main @ 516636f26 (9.0.0-dev) for the zstd token
Severity, scored against your policy: HIGH (Tier 1 feature enabled by default, broad evasion)

1. What happens

When an HTTP/1 message carries more than one value in Content-Encoding, libhtp-rs can build its
decompression chain with a layer missing, and then go ahead and run that chain. Suricata ends up
inspecting bytes that are neither the plaintext nor what was on the wire, but whatever is left after
the outer layers were undone, and it hands those bytes to the body hooks as if they were the body.

Content-Encoding: br, gzip over a body of gzip(br(P)) is the shortest example:

the header has two values, so libhtp-rs leaves its fast path and re-parses the value as a list
that list path recognises gzip, deflate, lzma, inflate, none and identity
it does not recognise br
br therefore resolves to HtpContentEncoding::None, and None is skipped when the chain is built
gzip gets undone, brotli does not

The body hooks then receive the 8,514-byte brotli stream instead of the 65,536-byte plaintext. Every
rule looking at http.response_body, http.request_body or file_data is dead for that transaction.
Nothing Suricata writes tells an operator that this happened. There is an applayer anomaly and a
shipped signature that watches it, and both are described in §4.

The scale matters here, because "one rule evaded" would understate it. On the affected transaction
the entire body-inspection surface is gone at once, in both directions, and neither end needs
anything beyond the ability to send a legal header. That is also the traffic class body rules exist
for: two attacker-controlled endpoints talking to each other.

2. Why it happens

2.1 The fast path and the list path disagree about what they know

response_initialize_decompressors() first tries to match the whole header value against a handful of
literals. Only if that fails does it set slow_path = true and re-parse the value as a
comma/space-separated list.

Suricata 8.0.7, rust/htp/src/response.rs, fast path (:1150-1158):

} else if ce.cmp_nocase_nozero(b"br") {          // :1153
    HtpContentEncoding::Brotli                   // :1154
} else if ce.cmp_nocase_nozero(b"inflate") || ce.cmp_nocase_nozero(b"none") {
    HtpContentEncoding::None
} else {
    slow_path = true;                            // :1158
    HtpContentEncoding::None
}

The same file, the list path's token table (:1183-1253):

for encoding in ce.split(|c| *c == b',' || *c == b' ') {     // :1183
    ...
    let encoding = if encoding.index_of_nocase(b"gzip").is_some() {
        ... HtpContentEncoding::Gzip
    } else if encoding.index_of_nocase(b"deflate").is_some() {
        ... HtpContentEncoding::Deflate
    } else if encoding.cmp_slice(b"lzma") == Ordering::Equal {
        ... HtpContentEncoding::Lzma
    } else if encoding.cmp_slice(b"inflate") == Ordering::Equal
           || encoding.cmp_slice(b"none")    == Ordering::Equal
           || encoding.cmp_slice(b"identity")== Ordering::Equal {
        HtpContentEncoding::None
    } else {
        htp_warn!(self.logger, HtpLogCode::ABNORMAL_CE_HEADER,
                  "C-E unknown setting");                    // :1246-1247
        HtpContentEncoding::None
    };
    self.response_prepend_decompressor(encoding)?;            // :1252
}

br is in the first table and not in the second, so it falls into the else branch. This is not a
typo in one place: each time an encoding was added to the fast path, the list path was left alone.
a1ff7424e added br, and on main 103abc2d3 added zstd (response.rs:1156, request.rs:1138)
without touching the list either. The two tables have quietly drifted apart.

2.2 `None` is skipped, so the chain is built short and then used

response_prepend_decompressor() does nothing for None (rust/htp/src/response.rs:1327-1333):

fn response_prepend_decompressor(&mut self, encoding: HtpContentEncoding) -> Result<()> {
    ...
    if encoding != HtpContentEncoding::None {
        ...   // push a decompressor
    }
    Ok(())
}

So for Content-Encoding: br, gzip:

step	token	resolved to	what it does to the chain
1	`br`	`None` (plus an `ABNORMAL_CE_HEADER` warning)	skipped
2	`gzip`	`Gzip`	pushed

The chain contains only the gzip decompressor. It inflates the outer layer without error, leaving
br(P) behind, and br(P) is what reaches the body hooks. Nothing fails, nothing restarts, and the
only trace is one low-severity anomaly.

The part I'd call the real defect is that a partially decoded buffer is handed downstream at all.
A buffer that is neither ciphertext nor plaintext is worse than a buffer that was never touched,
because everything downstream (rules, file extraction, hashing, protocol parsers) treats a
successfully inflated stream as trusted content and has no way to tell it is not.

2.3 Both directions

rust/htp/src/request.rs is structurally identical: request_initialize_decompressors() at :1115,
fast path at :1134 (br) and :1138 (zstd on main), list path from :1176 with the same
"C-E unknown setting" at :1238.

Request decompression is not something an operator opts into. src/app-layer-htp.c:1948-1958 calls
htp_config_set_request_decompression(cfg_prec->cfg, 1) unconditionally, and response decompression
has no switch at all.

3. Affected versions

Version	Affected	Notes
`main` (9.0.0-dev) @ `516636f26`	Yes	Unknown tokens: `br`, `zstd`
8.0.7 and the rest of 8.0.x	Yes	Unknown token: `br` (8.0.x has no zstd decoder)
7.0.x	Not tested	7.0 does not vendor `rust/htp`; it links the archived C libhtp. I did not build or test it, so I make no claim either way.

The functions involved in both rows are the same pair: response_initialize_decompressors() in
rust/htp/src/response.rs and request_initialize_decompressors() in rust/htp/src/request.rs.

4. Impact

For an affected transaction, the buffers used by detection hold the inner compressed stream, so all of
the following stop working at the same moment:

http.response_body / http.request_body content: and pcre: rules
file_data rules and file extraction; fileinfo reports the inner stream's size and hash, not the payload's
file.magic, hash-based file blacklists, and anything else derived from the extracted file
downstream parsing and protocol detection that consumes the body

In IDS the traffic passes. In IPS there is no match to act on, so there is nothing to drop either.

Operator visibility is the weakest part, and it is easy to overstate in either direction, so here is
what I actually observed. The only trace of the condition is the http.abnormal_ce_header applayer
anomaly. sid:2221033 watches it:

alert http any any -> any any (
  msg:"SURICATA HTTP Request abnormal Content-Encoding header";
  flow:established,to_server; app-layer-event:http.abnormal_ce_header;
  flowint:http.anomaly.count,+,1; classtype:protocol-command-decode; sid:2221033; rev:1;)

Two things about that rule are worth your attention. First, it fired on response-only flows in my
testing even though it is written to_server (the flow in question carried a single HTTP transaction,
a 200 response, no request-direction event; see §6.4). It looks like the anomaly is raised at flow
level rather than attached to a direction, so the to_server qualifier does not do what it reads
like. Second, and more importantly, the alert says "abnormal Content-Encoding header" and nothing
else. An operator who sees it learns that a peer sent an unusual header. Nothing there says that
every body signature on that transaction has stopped matching, and by the time this alert is worth
looking at, the transaction it describes is long over. For the response direction, where most body
signatures live, that is the whole of the signal.

5. Reproducer

5.1 The pcap

f2-repro.pcap holds four HTTP response transactions whose plaintext bodies are byte-identical and
which differ only in Content-Encoding and the matching encoding of the body:

#	request URI	`Content-Encoding`	body
1	`/vg/resp/plain`	(absent)	plaintext
2	`/vg/resp/gzip`	`gzip`	`gzip(P)`
3	`/vg/resp/br`	`br`	`br(P)`
4	`/vg/resp/br-gzip`	`br, gzip`	`gzip(br(P))`

102 packets, 99,937 bytes
sha256 4b0422fe75f5dddcde717812dfe48fb253106436603fcd0179b974e42aab1b53

The sessions are synthesised rather than captured. Each has a full three-way handshake, a request, the
complete response body and an orderly teardown, generated by make-pcap.py (appendix A). My first
attempt was a live tcpdump capture of the same traffic and it came out truncated, with every
response body cut roughly in half; Suricata could not reassemble complete bodies, alerted on nothing
at all, and the run looked exactly like "this does not reproduce". Appendix B5 has the details. The
synthesised pcap removes that variable.

Treat the pcap as non-confidential. It can go into your public CI or QA.

5.2 The four bodies are the same after decoding

The plaintext P is 65,536 bytes: a short canary block, compressible filler, and 8 KiB of
non-compressible random data. sha256(P) = 6453595dd48ae71437fa25b85ed02fe6779edd1d4f915d185dda5dead62f8665.
Every variant decodes back to exactly P, checked out of band with the corresponding decoders. All
four messages are valid and a compliant client decodes each of them to the same bytes.

The canaries sit at offset 0 of P, so they are always inside the default 100 KiB body limits.

5.3 Replaying it

The quickest route is the container image, which needs nothing installed:

docker run --rm --entrypoint /usr/bin/suricata -v $PWD:/d jasonish/suricata:8.0.7 \
    -r /d/input.pcap -S /d/test.rules -l /d/out -k none

input.pcap is the same file as f2-repro.pcap; test.rules is the single signature from §8. On
8.0.7 this prints read 1 file, 102 packets and writes three alerts to out/fast.log, not four.
Verbatim output and EVE are in evidence/docker-repro.txt and evidence/docker-repro-eve.json.

To build the pcap from scratch instead:

python3 make-pcap.py --payloads payloads-f2 --out input.pcap

6. Evidence

6.1 The reproducer, replayed offline

Running the signature from §8 over f2-repro.pcap on Suricata 8.0.7:

URI	`Content-Encoding`	`http.length` (wire)	`fileinfo.size` (inspected)	inspected bytes are plaintext?	alert
`/vg/resp/plain`	(absent)	65,536	65,536	yes	matched
`/vg/resp/gzip`	`gzip`	8,820	65,536	yes	matched
`/vg/resp/br`	`br`	8,514	65,536	yes	matched
`/vg/resp/br-gzip`	`br, gzip`	8,506	8,514	no, it is the inner brotli stream	none (+ `ABNORMAL_CE_HEADER`)

Three alerts on a pcap whose four transactions are identical once decoded.

This table also pins down a trap I fell into early on: http.length is the wire body length
(htp_tx_response_message_len, src/output-json-http.c:307) and tells you nothing about whether
inspection worked. It reads 8,820 for the healthy gzip case and 8,506 for the broken br, gzip
case, so it is lower when things are wrong. fileinfo.size is the useful signal, because file
extraction happens after decompression.

6.2 The same thing on a live instance

Offline replay proves the parsing defect. I also wanted to know whether it survives contact with a
real deployment, so I ran it against a public instance of mine: Suricata 8.0.7 RELEASE, af-packet on
eth0, with the stock ET Open ruleset loaded, 52,901 rules. I changed no configuration and added no
rules; the observation used signatures that were already in the loaded ruleset, which is why the
result below says something about a default install.

Signals recorded:

fileinfo.size, the number of bytes that reached the inspection pipeline. Needs no rules.
whether the real ET signature on the body matched
the ABNORMAL_CE_HEADER anomaly, which shows the unknown-token branch was taken

Response direction:

variant	`Content-Encoding`	plaintext alert	anomaly	`fileinfo.size`	verdict
`plain`	(absent)	matched (2045883)	none	65,536	normal
`gzip`	`gzip`	matched	none	65,536	normal
`br`	`br`	matched	none	65,536	normal
`gzip-gzip`	`gzip, gzip`	matched	none	65,536	normal
`br-space`	`br`	matched	none	65,536	normal
`br-gzip`	`br, gzip`	no match	`ABNORMAL_CE_HEADER`	8,514	evaded
`br-deflate`	`br, deflate`	no match	`ABNORMAL_CE_HEADER`	8,514	evaded
`gzip-br`	`gzip, br`	no match	`ABNORMAL_CE_HEADER`	8,561	evaded

Request direction, same layout:

variant	`Content-Encoding`	plaintext alert	`fileinfo.size`	verdict
`plain`	(absent)	matched (6 signatures)	65,536	normal
`gzip`	`gzip`	matched	65,536	normal
`br`	`br`	matched	65,536	normal
`gzip-gzip`	`gzip, gzip`	matched	65,536	normal
`br-gzip`	`br, gzip`	no match	8,514	evaded
`br-deflate`	`br, deflate`	no match	8,514	evaded

All 14 variants behaved as predicted. br-space is in there as a sanity check that a single br
with trailing whitespace is trimmed and takes the fast path, i.e. that the trigger is genuinely the
multi-value list and not whitespace.

Note the controls: gzip, br on its own, and gzip, gzip. Multi-layer decoding works fine when
every token is known, so the list path itself is not broken. Only the missing-token case is.

Formatted verdicts: evidence/f2-verify-result.txt. Raw EVE from that run:
evidence/f2-verify-raw-eve.txt.

6.3 The endpoints decode the same bytes without complaint

For the request-direction variants the test server decodes each received body using the layers named
in Content-Encoding and reports what it got. Every variant comes back as the full 65,536-byte
plaintext, br, gzip and br, deflate included (evidence/serve-request-log.txt). That rules out
"the encoding combination is malformed" as an explanation for the missing matches. The messages are
fine, both ends agree on what they mean, and Suricata is the only party that disagrees.

6.4 What the EVE records look like

/vg/resp/br-gzip, the evaded transaction:

{"event_type":"alert","src_ip":"115.200.248.4","dest_ip":"172.20.145.41","dest_port":8088,
 "alert":{"signature_id":2221033,"signature":"SURICATA HTTP Request abnormal Content-Encoding header"}}
{"event_type":"anomaly","anomaly":{"app_proto":"http","type":"applayer",
 "event":"ABNORMAL_CE_HEADER","layer":"proto_parser"}}
{"event_type":"fileinfo","src_ip":"172.20.145.41","src_port":8088,
 "fileinfo":{"filename":"vg-test.bin","state":"CLOSED","stored":false,"size":8514}}

size: 8514 is the inner brotli stream. The control /vg/resp/gzip reports 65536 and matches the
body signature. The single alert in that flow is sid:2221033, the anomaly rule. That flow holds one
HTTP transaction, the 200 response, and no request-direction event at all, yet the alert fired even
though the signature is written flow:established,to_server. Its message text talks about the request
direction, which does not help an operator arriving at it from the alert alone.

6.5 The regression test fails today, and that is the point

I ran the suricata-verify test from §8 under the same container image, using the official runner
(OISF/suricata-verify, run.py). It fails, in the expected way:

===> http-content-encoding-multivalue: Sub test #2: FAIL : expected 4 matches; got 3
     for filter {'count': 4, 'match': {'event_type': 'alert', 'alert.signature_id': 1000001}}

PASSED:  0
FAILED:  1
SKIPPED: 0

The first assertion (four HTTP transactions present) passes; the second (four alerts) does not. So the
test is red on 8.0.7 and should go green with the fix. Full runner output: evidence/sv-run.txt;
the Suricata invocation the runner built, the EVE and the fast log it produced:
evidence/sv-output-cmdline.txt, evidence/sv-output-eve.json, evidence/sv-output-fast.log.

7. Suggested fix

7.1 The minimum

Make the two tables one table: add br (and zstd on main) to the list path's token mapping in
both response.rs and request.rs. That covers everything I verified.

I would not stop there. Any encoding missing from the list path reproduces this behaviour, and the
set of encodings is not fixed; an attacker can send a private or extended token this code has never
heard of. Fixing the two tokens removes today's instances, not the class.

7.2 What I would prefer

Either of these, and I'm happy to prototype whichever you want:

One source of truth. Have the list path resolve each token through the same code the fast path
uses, so the two sets cannot diverge by construction.
Refuse to build a partial chain. If a token cannot be resolved, treat the message as
undecodable and do not pass a partially inflated buffer to the body hooks. Concretely: either skip
body detection for that transaction and record an event that says so, or fall back to delivering
the wire bytes with the buffer marked accordingly. Both are safer than the current mixture of
decoded and undecoded content.

The second option suggests a piece of hardening that stands on its own. Now that a body buffer can be
not-quite-plaintext, nothing downstream can tell. A buffer-level flag for "decompression incomplete,
do not trust this as plaintext" would let rule authors and file extraction make an informed choice
instead of a blind one.

7.3 Either way

If partial decoding stays possible in any form, it deserves a signal that says what it is. What fires
today reads as a request-side header complaint and says nothing about body coverage having been lost.
Something along the lines of "decompression incomplete, body detection skipped for this transaction"
would be actionable. What exists now is not.

8. Regression test

suricata-verify/http-content-encoding-multivalue/ next to this report contains test.yaml,
test.rules, input.pcap and README.md. Its input.pcap is the same file as
evidence/f2-repro.pcap.

It asserts 4 alerts for 4 transactions. On 8.0.7 it produces 3, so it is red before the fix and
green after. I'd suggest landing it together with the fix rather than before, so your CI does not go
red in the meantime.

9. Disclosure

Nothing here has been published. I searched the OISF/suricata tracker and the GitHub security
advisories and found no existing report or advisory for this behaviour.
Credit is welcome under my GitHub handle, Zer0Gate.
The pcap and the payloads are non-confidential and may enter public CI or QA.
All of this was done on instances and infrastructure I own.

Appendix A: files

In poc/suricata/round-03/ecs-verify/:

file	what it does
`gen-f2.py`	generates the body variants and `manifest.json`; checks each one by decoding it back and comparing with `P`
`serve.py`	test server; serves each variant with the matching `Content-Encoding`, and on request-direction runs decodes the received body and reports the result
`collect-f2.py`	parses EVE into the §6.2 verdicts
`make-pcap.py`	synthesises the reproducer: four complete HTTP/1.1 sessions, deterministic, no capture loss
`payloads-f2/`	the generated bodies

In evidence/:

file	what it is
`f2-repro.pcap`	the reproducer, 102 packets
`f2-repro.manifest.json`	sizes and hashes of the generated bodies
`docker-repro.txt`, `docker-repro-eve.json`, `docker-repro-fast.log`	the §5.3 replay and its output
`f2-verify-result.txt`	formatted verdicts from the §6.2 run
`f2-verify-raw-eve.txt`	raw EVE from the §6.2 run
`serve-request-log.txt`	endpoint decoding results, §6.3
`sv-run.txt`	the suricata-verify runner output from §6.5
`sv-output-cmdline.txt`, `sv-output-eve.json`, `sv-output-fast.log`, `sv-output-suricata.log`	what the runner's Suricata invocation produced

Appendix B: traps, in case they save you time

Each of these produced a convincing but wrong conclusion while I was working, and each would do the
same to anyone building a test on top of this bug.

B1. zstd proves nothing in this test. zstd emits RAW_BLOCKs, so literal plaintext, canary
included, ends up inside the zstd stream itself. A Content-Encoding: zstd, gzip body therefore
matches a plaintext signature whether or not Suricata drops the layer, and its fileinfo.size stays
near the compressed size either way. I first read that as "the bypass does not work on requests" and
it was an artefact. I dropped both zstd variants from the table for this reason. Brotli does not
have this property, which is why the br variants are the usable ones. Check:
gzip.decompress() of the zstd-gzip body still contains the canary; the same on br-gzip and
br-deflate does not.

B2. Direction matters as much as content. ET Open's C2 signatures mostly read
alert http $EXTERNAL_NET any -> $HOME_NET any (... to_client ...), on the assumption that the
client is inside and the C2 server is outside. In a test where the probed server is in $HOME_NET
and the tester is outside, the response direction runs $HOME_NET -> $EXTERNAL_NET and those rules
cannot fire. Easy to read as "evasion confirmed" when nothing was ever going to match. The
response-direction canaries in §6.2 were picked because their direction fits.

B3. flowbits, bsize and startswith preconditions. Several otherwise-simple body rules carry
a flowbits:isset,... prerequisite that a synthetic test will never satisfy. Three of my first
candidates had this problem.

B4. ET sid:2049403 and sid:2049404 cannot match on 8.0.7. They combine
content:"127.0.0.1"; content:"localhost|0a|"; fast_pattern; within:10; with
pcre:"/^127\.0\.0\.1\s+localhost[\r\n]+/m", which contradicts itself: 127.0.0.1 is 9 bytes and
the pcre needs at least one whitespace byte before localhost, so localhost\n can never start
before offset 10, while within:10 limits the scan to [0,10)
(detect-engine-content-inspection.c:180-181, depth = prev_buffer_offset + within + distance).
fast.log on the instance confirms neither has ever fired. Unrelated to this report; it just quietly
invalidated a control I had planned.

B5. A live capture can be truncated, and truncation looks like "no bug". My first reproducer was a
tcpdump -i eth0 -s 0 -w ... capture of exactly the traffic in §5.1. Every response body came out cut
roughly in half: 32,646 / 4,752 / 4,442 / 4,445 bytes instead of 65,536 / 8,820 / 8,514 / 8,506. With
bodies it could not reassemble, Suricata alerted on nothing, including the plaintext control, so that
run appeared to prove the opposite. If your control does not fire either, suspect the capture before
you suspect the finding.

B6. TCP flow control applies to hand-built pcaps too. My first synthesised pcap advertised a
29,200-byte receive window and then had the server push a 65,753-byte response in one go. Suricata,
correctly, refused to reassemble past the advertised window, so the plaintext control was truncated to
27,740 bytes while the three ~8.5 KB transactions were unaffected. That gives a partial evasion table
that looks like an artefact of body size rather than a Content-Encoding problem. make-pcap.py now
advertises 65,535 and inserts a client ACK between batches to reopen the window.

Appendix C: what I am not claiming

No memory-safety impact. This is a detection-coverage defect.
TLS is out of scope. Like any IDS without decryption, encrypted bodies are not inspectable, and
this report says nothing about that.
I did not test 7.0.x, and I make no claim about it.
I did not test IPS drop behaviour beyond noting that a rule which does not match cannot drop.
The body is equally invisible in IDS and IPS mode.

« Previous
1
2
3
…
5
Next »

(1-1/5)

Project

General

Profile

Suricata

Optimization #9173 » REPORT.md

Suricata: a multi-value `Content-Encoding` header silently disables HTTP body inspection

1. What happens

2. Why it happens

2.1 The fast path and the list path disagree about what they know

2.2 `None` is skipped, so the chain is built short and then used

2.3 Both directions

3. Affected versions

4. Impact

5. Reproducer

5.1 The pcap

5.2 The four bodies are the same after decoding

5.3 Replaying it

6. Evidence

6.1 The reproducer, replayed offline

6.2 The same thing on a live instance

6.3 The endpoints decode the same bytes without complaint

6.4 What the EVE records look like

6.5 The regression test fails today, and that is the point

7. Suggested fix

7.1 The minimum

7.2 What I would prefer

7.3 Either way

8. Regression test

9. Disclosure

Appendix A: files

Appendix B: traps, in case they save you time

Appendix C: what I am not claiming

Project

General

Profile

Suricata

Optimization #9173 » REPORT.md

Suricata: a multi-value Content-Encoding header silently disables HTTP body inspection

1. What happens

2. Why it happens

2.1 The fast path and the list path disagree about what they know

2.2 None is skipped, so the chain is built short and then used

2.3 Both directions

3. Affected versions

4. Impact

5. Reproducer

5.1 The pcap

5.2 The four bodies are the same after decoding

5.3 Replaying it

6. Evidence

6.1 The reproducer, replayed offline

6.2 The same thing on a live instance

6.3 The endpoints decode the same bytes without complaint

6.4 What the EVE records look like

6.5 The regression test fails today, and that is the point

7. Suggested fix

7.1 The minimum

7.2 What I would prefer

7.3 Either way

8. Regression test

9. Disclosure

Appendix A: files

Appendix B: traps, in case they save you time

Appendix C: what I am not claiming

Suricata: a multi-value `Content-Encoding` header silently disables HTTP body inspection

2.2 `None` is skipped, so the chain is built short and then used