Suricata: a multi-value Content-Encoding header silently disables HTTP body inspection

Reported by: Zer0Gate Date: 2026-09-24 Component: rust/htp (libhtp-rs), HTTP/1 body decompression Verified on: Suricata 8.0.7 RELEASE; main @ 516636f26 (9.0.0-dev) for the zstd token Severity, scored against your policy: HIGH (Tier 1 feature enabled by default, broad evasion)

1. What happens

When an HTTP/1 message carries more than one value in Content-Encoding, libhtp-rs can build its decompression chain with a layer missing, and then go ahead and run that chain. Suricata ends up inspecting bytes that are neither the plaintext nor what was on the wire, but whatever is left after the outer layers were undone, and it hands those bytes to the body hooks as if they were the body.

Content-Encoding: br, gzip over a body of gzip(br(P)) is the shortest example:

• the header has two values, so libhtp-rs leaves its fast path and re-parses the value as a list
• that list path recognises gzip, deflate, lzma, inflate, none and identity
• it does not recognise br
• br therefore resolves to HtpContentEncoding::None, and None is skipped when the chain is built
• gzip gets undone, brotli does not

The body hooks then receive the 8,514-byte brotli stream instead of the 65,536-byte plaintext. Every rule looking at http.response_body, http.request_body or file_data is dead for that transaction. Nothing Suricata writes tells an operator that this happened. There is an applayer anomaly and a shipped signature that watches it, and both are described in §4.

The scale matters here, because "one rule evaded" would understate it. On the affected transaction the entire body-inspection surface is gone at once, in both directions, and neither end needs anything beyond the ability to send a legal header. That is also the traffic class body rules exist for: two attacker-controlled endpoints talking to each other.

2. Why it happens

2.1 The fast path and the list path disagree about what they know

response_initialize_decompressors() first tries to match the whole header value against a handful of literals. Only if that fails does it set slow_path = true and re-parse the value as a comma/space-separated list.

Suricata 8.0.7, rust/htp/src/response.rs, fast path (:1150-1158):

} else if ce.cmp_nocase_nozero(b"br") {          // :1153
    HtpContentEncoding::Brotli                   // :1154
} else if ce.cmp_nocase_nozero(b"inflate") || ce.cmp_nocase_nozero(b"none") {
    HtpContentEncoding::None
} else {
    slow_path = true;                            // :1158
    HtpContentEncoding::None
}

The same file, the list path's token table (:1183-1253):

for encoding in ce.split(|c| *c == b',' || *c == b' ') {     // :1183
    ...
    let encoding = if encoding.index_of_nocase(b"gzip").is_some() {
        ... HtpContentEncoding::Gzip
    } else if encoding.index_of_nocase(b"deflate").is_some() {
        ... HtpContentEncoding::Deflate
    } else if encoding.cmp_slice(b"lzma") == Ordering::Equal {
        ... HtpContentEncoding::Lzma
    } else if encoding.cmp_slice(b"inflate") == Ordering::Equal
           || encoding.cmp_slice(b"none")    == Ordering::Equal
           || encoding.cmp_slice(b"identity")== Ordering::Equal {
        HtpContentEncoding::None
    } else {
        htp_warn!(self.logger, HtpLogCode::ABNORMAL_CE_HEADER,
                  "C-E unknown setting");                    // :1246-1247
        HtpContentEncoding::None
    };
    self.response_prepend_decompressor(encoding)?;            // :1252
}

br is in the first table and not in the second, so it falls into the else branch. This is not a typo in one place: each time an encoding was added to the fast path, the list path was left alone. a1ff7424e added br, and on main 103abc2d3 added zstd (response.rs:1156, request.rs:1138) without touching the list either. The two tables have quietly drifted apart.

2.2 None is skipped, so the chain is built short and then used

response_prepend_decompressor() does nothing for None (rust/htp/src/response.rs:1327-1333):

fn response_prepend_decompressor(&mut self, encoding: HtpContentEncoding) -> Result<()> {
    ...
    if encoding != HtpContentEncoding::None {
        ...   // push a decompressor
    }
    Ok(())
}

So for Content-Encoding: br, gzip:

The chain contains only the gzip decompressor. It inflates the outer layer without error, leaving br(P) behind, and br(P) is what reaches the body hooks. Nothing fails, nothing restarts, and the only trace is one low-severity anomaly.

The part I'd call the real defect is that a partially decoded buffer is handed downstream at all. A buffer that is neither ciphertext nor plaintext is worse than a buffer that was never touched, because everything downstream (rules, file extraction, hashing, protocol parsers) treats a successfully inflated stream as trusted content and has no way to tell it is not.

2.3 Both directions

rust/htp/src/request.rs is structurally identical: request_initialize_decompressors() at :1115, fast path at :1134 (br) and :1138 (zstd on main), list path from :1176 with the same "C-E unknown setting" at :1238.

Request decompression is not something an operator opts into. src/app-layer-htp.c:1948-1958 calls htp_config_set_request_decompression(cfg_prec->cfg, 1) unconditionally, and response decompression has no switch at all.

3. Affected versions

The functions involved in both rows are the same pair: response_initialize_decompressors() in rust/htp/src/response.rs and request_initialize_decompressors() in rust/htp/src/request.rs.

4. Impact

For an affected transaction, the buffers used by detection hold the inner compressed stream, so all of the following stop working at the same moment:

• http.response_body / http.request_body content: and pcre: rules
• file_data rules and file extraction; fileinfo reports the inner stream's size and hash, not the payload's
• file.magic, hash-based file blacklists, and anything else derived from the extracted file
• downstream parsing and protocol detection that consumes the body

In IDS the traffic passes. In IPS there is no match to act on, so there is nothing to drop either.

Operator visibility is the weakest part, and it is easy to overstate in either direction, so here is what I actually observed. The only trace of the condition is the http.abnormal_ce_header applayer anomaly. sid:2221033 watches it:

alert http any any -> any any (
  msg:"SURICATA HTTP Request abnormal Content-Encoding header";
  flow:established,to_server; app-layer-event:http.abnormal_ce_header;
  flowint:http.anomaly.count,+,1; classtype:protocol-command-decode; sid:2221033; rev:1;)

Two things about that rule are worth your attention. First, it fired on response-only flows in my testing even though it is written to_server (the flow in question carried a single HTTP transaction, a 200 response, no request-direction event; see §6.4). It looks like the anomaly is raised at flow level rather than attached to a direction, so the to_server qualifier does not do what it reads like. Second, and more importantly, the alert says "abnormal Content-Encoding header" and nothing else. An operator who sees it learns that a peer sent an unusual header. Nothing there says that every body signature on that transaction has stopped matching, and by the time this alert is worth looking at, the transaction it describes is long over. For the response direction, where most body signatures live, that is the whole of the signal.

5. Reproducer

5.1 The pcap

f2-repro.pcap holds four HTTP response transactions whose plaintext bodies are byte-identical and which differ only in Content-Encoding and the matching encoding of the body:

• 102 packets, 99,937 bytes
• sha256 4b0422fe75f5dddcde717812dfe48fb253106436603fcd0179b974e42aab1b53

The sessions are synthesised rather than captured. Each has a full three-way handshake, a request, the complete response body and an orderly teardown, generated by make-pcap.py (appendix A). My first attempt was a live tcpdump capture of the same traffic and it came out truncated, with every response body cut roughly in half; Suricata could not reassemble complete bodies, alerted on nothing at all, and the run looked exactly like "this does not reproduce". Appendix B5 has the details. The synthesised pcap removes that variable.

Treat the pcap as non-confidential. It can go into your public CI or QA.

5.2 The four bodies are the same after decoding

The plaintext P is 65,536 bytes: a short canary block, compressible filler, and 8 KiB of non-compressible random data. sha256(P) = 6453595dd48ae71437fa25b85ed02fe6779edd1d4f915d185dda5dead62f8665. Every variant decodes back to exactly P, checked out of band with the corresponding decoders. All four messages are valid and a compliant client decodes each of them to the same bytes.

The canaries sit at offset 0 of P, so they are always inside the default 100 KiB body limits.

5.3 Replaying it

The quickest route is the container image, which needs nothing installed:

docker run --rm --entrypoint /usr/bin/suricata -v $PWD:/d jasonish/suricata:8.0.7 \
    -r /d/input.pcap -S /d/test.rules -l /d/out -k none

input.pcap is the same file as f2-repro.pcap; test.rules is the single signature from §8. On 8.0.7 this prints read 1 file, 102 packets and writes three alerts to out/fast.log, not four. Verbatim output and EVE are in evidence/docker-repro.txt and evidence/docker-repro-eve.json.

To build the pcap from scratch instead:

python3 make-pcap.py --payloads payloads-f2 --out input.pcap

6. Evidence

6.1 The reproducer, replayed offline

Running the signature from §8 over f2-repro.pcap on Suricata 8.0.7:

Three alerts on a pcap whose four transactions are identical once decoded.

This table also pins down a trap I fell into early on: http.length is the wire body length (htp_tx_response_message_len, src/output-json-http.c:307) and tells you nothing about whether inspection worked. It reads 8,820 for the healthy gzip case and 8,506 for the broken br, gzip case, so it is lower when things are wrong. fileinfo.size is the useful signal, because file extraction happens after decompression.

6.2 The same thing on a live instance

Offline replay proves the parsing defect. I also wanted to know whether it survives contact with a real deployment, so I ran it against a public instance of mine: Suricata 8.0.7 RELEASE, af-packet on eth0, with the stock ET Open ruleset loaded, 52,901 rules. I changed no configuration and added no rules; the observation used signatures that were already in the loaded ruleset, which is why the result below says something about a default install.

Signals recorded:

• fileinfo.size, the number of bytes that reached the inspection pipeline. Needs no rules.
• whether the real ET signature on the body matched
• the ABNORMAL_CE_HEADER anomaly, which shows the unknown-token branch was taken

Response direction:

Request direction, same layout:

All 14 variants behaved as predicted. br-space is in there as a sanity check that a single br with trailing whitespace is trimmed and takes the fast path, i.e. that the trigger is genuinely the multi-value list and not whitespace.

Note the controls: gzip, br on its own, and gzip, gzip. Multi-layer decoding works fine when every token is known, so the list path itself is not broken. Only the missing-token case is.

Formatted verdicts: evidence/f2-verify-result.txt. Raw EVE from that run: evidence/f2-verify-raw-eve.txt.

6.3 The endpoints decode the same bytes without complaint

For the request-direction variants the test server decodes each received body using the layers named in Content-Encoding and reports what it got. Every variant comes back as the full 65,536-byte plaintext, br, gzip and br, deflate included (evidence/serve-request-log.txt). That rules out "the encoding combination is malformed" as an explanation for the missing matches. The messages are fine, both ends agree on what they mean, and Suricata is the only party that disagrees.

6.4 What the EVE records look like

/vg/resp/br-gzip, the evaded transaction:

{"event_type":"alert","src_ip":"115.200.248.4","dest_ip":"172.20.145.41","dest_port":8088,
 "alert":{"signature_id":2221033,"signature":"SURICATA HTTP Request abnormal Content-Encoding header"}}
{"event_type":"anomaly","anomaly":{"app_proto":"http","type":"applayer",
 "event":"ABNORMAL_CE_HEADER","layer":"proto_parser"}}
{"event_type":"fileinfo","src_ip":"172.20.145.41","src_port":8088,
 "fileinfo":{"filename":"vg-test.bin","state":"CLOSED","stored":false,"size":8514}}

size: 8514 is the inner brotli stream. The control /vg/resp/gzip reports 65536 and matches the body signature. The single alert in that flow is sid:2221033, the anomaly rule. That flow holds one HTTP transaction, the 200 response, and no request-direction event at all, yet the alert fired even though the signature is written flow:established,to_server. Its message text talks about the request direction, which does not help an operator arriving at it from the alert alone.

6.5 The regression test fails today, and that is the point

I ran the suricata-verify test from §8 under the same container image, using the official runner (OISF/suricata-verify, run.py). It fails, in the expected way:

===> http-content-encoding-multivalue: Sub test #2: FAIL : expected 4 matches; got 3
     for filter {'count': 4, 'match': {'event_type': 'alert', 'alert.signature_id': 1000001}}

PASSED:  0
FAILED:  1
SKIPPED: 0

The first assertion (four HTTP transactions present) passes; the second (four alerts) does not. So the test is red on 8.0.7 and should go green with the fix. Full runner output: evidence/sv-run.txt; the Suricata invocation the runner built, the EVE and the fast log it produced: evidence/sv-output-cmdline.txt, evidence/sv-output-eve.json, evidence/sv-output-fast.log.

7. Suggested fix

7.1 The minimum

Make the two tables one table: add br (and zstd on main) to the list path's token mapping in both response.rs and request.rs. That covers everything I verified.

I would not stop there. Any encoding missing from the list path reproduces this behaviour, and the set of encodings is not fixed; an attacker can send a private or extended token this code has never heard of. Fixing the two tokens removes today's instances, not the class.

7.2 What I would prefer

Either of these, and I'm happy to prototype whichever you want:

1. One source of truth. Have the list path resolve each token through the same code the fast path

uses, so the two sets cannot diverge by construction.

1. Refuse to build a partial chain. If a token cannot be resolved, treat the message as

undecodable and do not pass a partially inflated buffer to the body hooks. Concretely: either skip body detection for that transaction and record an event that says so, or fall back to delivering the wire bytes with the buffer marked accordingly. Both are safer than the current mixture of decoded and undecoded content.

The second option suggests a piece of hardening that stands on its own. Now that a body buffer can be not-quite-plaintext, nothing downstream can tell. A buffer-level flag for "decompression incomplete, do not trust this as plaintext" would let rule authors and file extraction make an informed choice instead of a blind one.

7.3 Either way

If partial decoding stays possible in any form, it deserves a signal that says what it is. What fires today reads as a request-side header complaint and says nothing about body coverage having been lost. Something along the lines of "decompression incomplete, body detection skipped for this transaction" would be actionable. What exists now is not.

8. Regression test

suricata-verify/http-content-encoding-multivalue/ next to this report contains test.yaml, test.rules, input.pcap and README.md. Its input.pcap is the same file as evidence/f2-repro.pcap.

It asserts 4 alerts for 4 transactions. On 8.0.7 it produces 3, so it is red before the fix and green after. I'd suggest landing it together with the fix rather than before, so your CI does not go red in the meantime.

9. Disclosure

• Nothing here has been published. I searched the OISF/suricata tracker and the GitHub security

advisories and found no existing report or advisory for this behaviour.

• Credit is welcome under my GitHub handle, Zer0Gate.
• The pcap and the payloads are non-confidential and may enter public CI or QA.
• All of this was done on instances and infrastructure I own.

Appendix A: files

In poc/suricata/round-03/ecs-verify/:

In evidence/:

Appendix B: traps, in case they save you time

Each of these produced a convincing but wrong conclusion while I was working, and each would do the same to anyone building a test on top of this bug.

B1. zstd proves nothing in this test. zstd emits RAW_BLOCKs, so literal plaintext, canary included, ends up inside the zstd stream itself. A Content-Encoding: zstd, gzip body therefore matches a plaintext signature whether or not Suricata drops the layer, and its fileinfo.size stays near the compressed size either way. I first read that as "the bypass does not work on requests" and it was an artefact. I dropped both zstd variants from the table for this reason. Brotli does not have this property, which is why the br variants are the usable ones. Check: gzip.decompress() of the zstd-gzip body still contains the canary; the same on br-gzip and br-deflate does not.

B2. Direction matters as much as content. ET Open's C2 signatures mostly read alert http $EXTERNAL_NET any -> $HOME_NET any (... to_client ...), on the assumption that the client is inside and the C2 server is outside. In a test where the probed server is in $HOME_NET and the tester is outside, the response direction runs $HOME_NET -> $EXTERNAL_NET and those rules cannot fire. Easy to read as "evasion confirmed" when nothing was ever going to match. The response-direction canaries in §6.2 were picked because their direction fits.

B3. flowbits, bsize and startswith preconditions. Several otherwise-simple body rules carry a flowbits:isset,... prerequisite that a synthetic test will never satisfy. Three of my first candidates had this problem.

B4. ET sid:2049403 and sid:2049404 cannot match on 8.0.7. They combine content:"127.0.0.1"; content:"localhost|0a|"; fast_pattern; within:10; with pcre:"/^127\.0\.0\.1\s+localhost[\r\n]+/m", which contradicts itself: 127.0.0.1 is 9 bytes and the pcre needs at least one whitespace byte before localhost, so localhost\n can never start before offset 10, while within:10 limits the scan to [0,10) (detect-engine-content-inspection.c:180-181, depth = prev_buffer_offset + within + distance). fast.log on the instance confirms neither has ever fired. Unrelated to this report; it just quietly invalidated a control I had planned.

B5. A live capture can be truncated, and truncation looks like "no bug". My first reproducer was a tcpdump -i eth0 -s 0 -w ... capture of exactly the traffic in §5.1. Every response body came out cut roughly in half: 32,646 / 4,752 / 4,442 / 4,445 bytes instead of 65,536 / 8,820 / 8,514 / 8,506. With bodies it could not reassemble, Suricata alerted on nothing, including the plaintext control, so that run appeared to prove the opposite. If your control does not fire either, suspect the capture before you suspect the finding.

B6. TCP flow control applies to hand-built pcaps too. My first synthesised pcap advertised a 29,200-byte receive window and then had the server push a 65,753-byte response in one go. Suricata, correctly, refused to reassemble past the advertised window, so the plaintext control was truncated to 27,740 bytes while the three ~8.5 KB transactions were unaffected. That gives a partial evasion table that looks like an artefact of body size rather than a Content-Encoding problem. make-pcap.py now advertises 65,535 and inserts a client ACK between batches to reopen the window.

Appendix C: what I am not claiming

• No memory-safety impact. This is a detection-coverage defect.
• TLS is out of scope. Like any IDS without decryption, encrypted bodies are not inspectable, and

this report says nothing about that.

• I did not test 7.0.x, and I make no claim about it.
• I did not test IPS drop behaviour beyond noting that a rule which does not match cannot drop.

The body is equally invisible in IDS and IPS mode.