# Suricata: a multi-value `Content-Encoding` header silently disables HTTP body inspection **Reported by:** Zer0Gate **Date:** 2026-09-24 **Component:** `rust/htp` (libhtp-rs), HTTP/1 body decompression **Verified on:** Suricata 8.0.7 RELEASE; `main` @ `516636f26` (9.0.0-dev) for the `zstd` token **Severity, scored against your policy:** HIGH (Tier 1 feature enabled by default, broad evasion) --- ## 1. What happens When an HTTP/1 message carries more than one value in `Content-Encoding`, libhtp-rs can build its decompression chain with a layer missing, and then go ahead and run that chain. Suricata ends up inspecting bytes that are neither the plaintext nor what was on the wire, but whatever is left after the outer layers were undone, and it hands those bytes to the body hooks as if they were the body. `Content-Encoding: br, gzip` over a body of `gzip(br(P))` is the shortest example: * the header has two values, so libhtp-rs leaves its fast path and re-parses the value as a list * that list path recognises `gzip`, `deflate`, `lzma`, `inflate`, `none` and `identity` * it does not recognise `br` * `br` therefore resolves to `HtpContentEncoding::None`, and `None` is skipped when the chain is built * gzip gets undone, brotli does not The body hooks then receive the 8,514-byte brotli stream instead of the 65,536-byte plaintext. Every rule looking at `http.response_body`, `http.request_body` or `file_data` is dead for that transaction. Nothing Suricata writes tells an operator that this happened. There is an applayer anomaly and a shipped signature that watches it, and both are described in §4. The scale matters here, because "one rule evaded" would understate it. On the affected transaction the entire body-inspection surface is gone at once, in both directions, and neither end needs anything beyond the ability to send a legal header. That is also the traffic class body rules exist for: two attacker-controlled endpoints talking to each other. ## 2. Why it happens ### 2.1 The fast path and the list path disagree about what they know `response_initialize_decompressors()` first tries to match the whole header value against a handful of literals. Only if that fails does it set `slow_path = true` and re-parse the value as a comma/space-separated list. Suricata 8.0.7, `rust/htp/src/response.rs`, fast path (`:1150-1158`): ```rust } else if ce.cmp_nocase_nozero(b"br") { // :1153 HtpContentEncoding::Brotli // :1154 } else if ce.cmp_nocase_nozero(b"inflate") || ce.cmp_nocase_nozero(b"none") { HtpContentEncoding::None } else { slow_path = true; // :1158 HtpContentEncoding::None } ``` The same file, the list path's token table (`:1183-1253`): ```rust for encoding in ce.split(|c| *c == b',' || *c == b' ') { // :1183 ... let encoding = if encoding.index_of_nocase(b"gzip").is_some() { ... HtpContentEncoding::Gzip } else if encoding.index_of_nocase(b"deflate").is_some() { ... HtpContentEncoding::Deflate } else if encoding.cmp_slice(b"lzma") == Ordering::Equal { ... HtpContentEncoding::Lzma } else if encoding.cmp_slice(b"inflate") == Ordering::Equal || encoding.cmp_slice(b"none") == Ordering::Equal || encoding.cmp_slice(b"identity")== Ordering::Equal { HtpContentEncoding::None } else { htp_warn!(self.logger, HtpLogCode::ABNORMAL_CE_HEADER, "C-E unknown setting"); // :1246-1247 HtpContentEncoding::None }; self.response_prepend_decompressor(encoding)?; // :1252 } ``` `br` is in the first table and not in the second, so it falls into the `else` branch. This is not a typo in one place: each time an encoding was added to the fast path, the list path was left alone. `a1ff7424e` added `br`, and on `main` `103abc2d3` added `zstd` (`response.rs:1156`, `request.rs:1138`) without touching the list either. The two tables have quietly drifted apart. ### 2.2 `None` is skipped, so the chain is built short and then used `response_prepend_decompressor()` does nothing for `None` (`rust/htp/src/response.rs:1327-1333`): ```rust fn response_prepend_decompressor(&mut self, encoding: HtpContentEncoding) -> Result<()> { ... if encoding != HtpContentEncoding::None { ... // push a decompressor } Ok(()) } ``` So for `Content-Encoding: br, gzip`: | step | token | resolved to | what it does to the chain | |---|---|---|---| | 1 | `br` | `None` (plus an `ABNORMAL_CE_HEADER` warning) | skipped | | 2 | `gzip` | `Gzip` | pushed | The chain contains only the gzip decompressor. It inflates the outer layer without error, leaving `br(P)` behind, and `br(P)` is what reaches the body hooks. Nothing fails, nothing restarts, and the only trace is one low-severity anomaly. The part I'd call the real defect is that a partially decoded buffer is handed downstream at all. A buffer that is neither ciphertext nor plaintext is worse than a buffer that was never touched, because everything downstream (rules, file extraction, hashing, protocol parsers) treats a successfully inflated stream as trusted content and has no way to tell it is not. ### 2.3 Both directions `rust/htp/src/request.rs` is structurally identical: `request_initialize_decompressors()` at `:1115`, fast path at `:1134` (`br`) and `:1138` (`zstd` on `main`), list path from `:1176` with the same `"C-E unknown setting"` at `:1238`. Request decompression is not something an operator opts into. `src/app-layer-htp.c:1948-1958` calls `htp_config_set_request_decompression(cfg_prec->cfg, 1)` unconditionally, and response decompression has no switch at all. ## 3. Affected versions | Version | Affected | Notes | |---|---|---| | `main` (9.0.0-dev) @ `516636f26` | Yes | Unknown tokens: `br`, `zstd` | | 8.0.7 and the rest of 8.0.x | Yes | Unknown token: `br` (8.0.x has no zstd decoder) | | 7.0.x | Not tested | 7.0 does not vendor `rust/htp`; it links the archived C libhtp. I did not build or test it, so I make no claim either way. | The functions involved in both rows are the same pair: `response_initialize_decompressors()` in `rust/htp/src/response.rs` and `request_initialize_decompressors()` in `rust/htp/src/request.rs`. ## 4. Impact For an affected transaction, the buffers used by detection hold the inner compressed stream, so all of the following stop working at the same moment: * `http.response_body` / `http.request_body` `content:` and `pcre:` rules * `file_data` rules and file extraction; `fileinfo` reports the inner stream's size and hash, not the payload's * `file.magic`, hash-based file blacklists, and anything else derived from the extracted file * downstream parsing and protocol detection that consumes the body In IDS the traffic passes. In IPS there is no match to act on, so there is nothing to drop either. Operator visibility is the weakest part, and it is easy to overstate in either direction, so here is what I actually observed. The only trace of the condition is the `http.abnormal_ce_header` applayer anomaly. `sid:2221033` watches it: ``` alert http any any -> any any ( msg:"SURICATA HTTP Request abnormal Content-Encoding header"; flow:established,to_server; app-layer-event:http.abnormal_ce_header; flowint:http.anomaly.count,+,1; classtype:protocol-command-decode; sid:2221033; rev:1;) ``` Two things about that rule are worth your attention. First, it fired on response-only flows in my testing even though it is written `to_server` (the flow in question carried a single HTTP transaction, a 200 response, no request-direction event; see §6.4). It looks like the anomaly is raised at flow level rather than attached to a direction, so the `to_server` qualifier does not do what it reads like. Second, and more importantly, the alert says "abnormal Content-Encoding header" and nothing else. An operator who sees it learns that a peer sent an unusual header. Nothing there says that every body signature on that transaction has stopped matching, and by the time this alert is worth looking at, the transaction it describes is long over. For the response direction, where most body signatures live, that is the whole of the signal. ## 5. Reproducer ### 5.1 The pcap `f2-repro.pcap` holds four HTTP response transactions whose plaintext bodies are byte-identical and which differ only in `Content-Encoding` and the matching encoding of the body: | # | request URI | `Content-Encoding` | body | |---|---|---|---| | 1 | `/vg/resp/plain` | *(absent)* | plaintext | | 2 | `/vg/resp/gzip` | `gzip` | `gzip(P)` | | 3 | `/vg/resp/br` | `br` | `br(P)` | | 4 | `/vg/resp/br-gzip` | `br, gzip` | `gzip(br(P))` | * 102 packets, 99,937 bytes * sha256 `4b0422fe75f5dddcde717812dfe48fb253106436603fcd0179b974e42aab1b53` The sessions are synthesised rather than captured. Each has a full three-way handshake, a request, the complete response body and an orderly teardown, generated by `make-pcap.py` (appendix A). My first attempt was a live `tcpdump` capture of the same traffic and it came out truncated, with every response body cut roughly in half; Suricata could not reassemble complete bodies, alerted on nothing at all, and the run looked exactly like "this does not reproduce". Appendix B5 has the details. The synthesised pcap removes that variable. Treat the pcap as **non-confidential**. It can go into your public CI or QA. ### 5.2 The four bodies are the same after decoding The plaintext `P` is 65,536 bytes: a short canary block, compressible filler, and 8 KiB of non-compressible random data. `sha256(P) = 6453595dd48ae71437fa25b85ed02fe6779edd1d4f915d185dda5dead62f8665`. Every variant decodes back to exactly `P`, checked out of band with the corresponding decoders. All four messages are valid and a compliant client decodes each of them to the same bytes. The canaries sit at offset 0 of `P`, so they are always inside the default 100 KiB body limits. ### 5.3 Replaying it The quickest route is the container image, which needs nothing installed: ``` docker run --rm --entrypoint /usr/bin/suricata -v $PWD:/d jasonish/suricata:8.0.7 \ -r /d/input.pcap -S /d/test.rules -l /d/out -k none ``` `input.pcap` is the same file as `f2-repro.pcap`; `test.rules` is the single signature from §8. On 8.0.7 this prints `read 1 file, 102 packets` and writes three alerts to `out/fast.log`, not four. Verbatim output and EVE are in `evidence/docker-repro.txt` and `evidence/docker-repro-eve.json`. To build the pcap from scratch instead: ``` python3 make-pcap.py --payloads payloads-f2 --out input.pcap ``` ## 6. Evidence ### 6.1 The reproducer, replayed offline Running the signature from §8 over `f2-repro.pcap` on Suricata 8.0.7: | URI | `Content-Encoding` | `http.length` (wire) | `fileinfo.size` (inspected) | inspected bytes are plaintext? | alert | |---|---|---|---|---|---| | `/vg/resp/plain` | *(absent)* | 65,536 | **65,536** | yes | **matched** | | `/vg/resp/gzip` | `gzip` | 8,820 | **65,536** | yes | **matched** | | `/vg/resp/br` | `br` | 8,514 | **65,536** | yes | **matched** | | **`/vg/resp/br-gzip`** | **`br, gzip`** | 8,506 | **8,514** | **no, it is the inner brotli stream** | **none** (+ `ABNORMAL_CE_HEADER`) | Three alerts on a pcap whose four transactions are identical once decoded. This table also pins down a trap I fell into early on: `http.length` is the **wire** body length (`htp_tx_response_message_len`, `src/output-json-http.c:307`) and tells you nothing about whether inspection worked. It reads 8,820 for the healthy `gzip` case and 8,506 for the broken `br, gzip` case, so it is lower when things are wrong. `fileinfo.size` is the useful signal, because file extraction happens after decompression. ### 6.2 The same thing on a live instance Offline replay proves the parsing defect. I also wanted to know whether it survives contact with a real deployment, so I ran it against a public instance of mine: Suricata 8.0.7 RELEASE, af-packet on `eth0`, with the stock ET Open ruleset loaded, 52,901 rules. I changed no configuration and added no rules; the observation used signatures that were already in the loaded ruleset, which is why the result below says something about a default install. Signals recorded: * **`fileinfo.size`**, the number of bytes that reached the inspection pipeline. Needs no rules. * whether the real ET signature on the body matched * the `ABNORMAL_CE_HEADER` anomaly, which shows the unknown-token branch was taken Response direction: | variant | `Content-Encoding` | plaintext alert | anomaly | `fileinfo.size` | verdict | |---|---|---|---|---|---| | `plain` | *(absent)* | **matched** (2045883) | none | 65,536 | normal | | `gzip` | `gzip` | **matched** | none | 65,536 | normal | | `br` | `br` | **matched** | none | 65,536 | normal | | `gzip-gzip` | `gzip, gzip` | **matched** | none | 65,536 | normal | | `br-space` | `br ` | **matched** | none | 65,536 | normal | | **`br-gzip`** | **`br, gzip`** | **no match** | `ABNORMAL_CE_HEADER` | **8,514** | **evaded** | | **`br-deflate`** | **`br, deflate`** | **no match** | `ABNORMAL_CE_HEADER` | **8,514** | **evaded** | | **`gzip-br`** | **`gzip, br`** | **no match** | `ABNORMAL_CE_HEADER` | **8,561** | **evaded** | Request direction, same layout: | variant | `Content-Encoding` | plaintext alert | `fileinfo.size` | verdict | |---|---|---|---|---| | `plain` | *(absent)* | **matched** (6 signatures) | 65,536 | normal | | `gzip` | `gzip` | **matched** | 65,536 | normal | | `br` | `br` | **matched** | 65,536 | normal | | `gzip-gzip` | `gzip, gzip` | **matched** | 65,536 | normal | | **`br-gzip`** | **`br, gzip`** | **no match** | **8,514** | **evaded** | | **`br-deflate`** | **`br, deflate`** | **no match** | **8,514** | **evaded** | All 14 variants behaved as predicted. `br-space` is in there as a sanity check that a single `br` with trailing whitespace is trimmed and takes the fast path, i.e. that the trigger is genuinely the multi-value list and not whitespace. Note the controls: `gzip`, `br` on its own, and `gzip, gzip`. Multi-layer decoding works fine when every token is known, so the list path itself is not broken. Only the missing-token case is. Formatted verdicts: `evidence/f2-verify-result.txt`. Raw EVE from that run: `evidence/f2-verify-raw-eve.txt`. ### 6.3 The endpoints decode the same bytes without complaint For the request-direction variants the test server decodes each received body using the layers named in `Content-Encoding` and reports what it got. Every variant comes back as the full 65,536-byte plaintext, `br, gzip` and `br, deflate` included (`evidence/serve-request-log.txt`). That rules out "the encoding combination is malformed" as an explanation for the missing matches. The messages are fine, both ends agree on what they mean, and Suricata is the only party that disagrees. ### 6.4 What the EVE records look like `/vg/resp/br-gzip`, the evaded transaction: ```json {"event_type":"alert","src_ip":"115.200.248.4","dest_ip":"172.20.145.41","dest_port":8088, "alert":{"signature_id":2221033,"signature":"SURICATA HTTP Request abnormal Content-Encoding header"}} {"event_type":"anomaly","anomaly":{"app_proto":"http","type":"applayer", "event":"ABNORMAL_CE_HEADER","layer":"proto_parser"}} {"event_type":"fileinfo","src_ip":"172.20.145.41","src_port":8088, "fileinfo":{"filename":"vg-test.bin","state":"CLOSED","stored":false,"size":8514}} ``` `size: 8514` is the inner brotli stream. The control `/vg/resp/gzip` reports `65536` and matches the body signature. The single alert in that flow is `sid:2221033`, the anomaly rule. That flow holds one HTTP transaction, the 200 response, and no request-direction event at all, yet the alert fired even though the signature is written `flow:established,to_server`. Its message text talks about the request direction, which does not help an operator arriving at it from the alert alone. ### 6.5 The regression test fails today, and that is the point I ran the `suricata-verify` test from §8 under the same container image, using the official runner (`OISF/suricata-verify`, `run.py`). It fails, in the expected way: ``` ===> http-content-encoding-multivalue: Sub test #2: FAIL : expected 4 matches; got 3 for filter {'count': 4, 'match': {'event_type': 'alert', 'alert.signature_id': 1000001}} PASSED: 0 FAILED: 1 SKIPPED: 0 ``` The first assertion (four HTTP transactions present) passes; the second (four alerts) does not. So the test is red on 8.0.7 and should go green with the fix. Full runner output: `evidence/sv-run.txt`; the Suricata invocation the runner built, the EVE and the fast log it produced: `evidence/sv-output-cmdline.txt`, `evidence/sv-output-eve.json`, `evidence/sv-output-fast.log`. ## 7. Suggested fix ### 7.1 The minimum Make the two tables one table: add `br` (and `zstd` on `main`) to the list path's token mapping in both `response.rs` and `request.rs`. That covers everything I verified. I would not stop there. Any encoding missing from the list path reproduces this behaviour, and the set of encodings is not fixed; an attacker can send a private or extended token this code has never heard of. Fixing the two tokens removes today's instances, not the class. ### 7.2 What I would prefer Either of these, and I'm happy to prototype whichever you want: 1. **One source of truth.** Have the list path resolve each token through the same code the fast path uses, so the two sets cannot diverge by construction. 2. **Refuse to build a partial chain.** If a token cannot be resolved, treat the message as undecodable and do not pass a partially inflated buffer to the body hooks. Concretely: either skip body detection for that transaction and record an event that says so, or fall back to delivering the wire bytes with the buffer marked accordingly. Both are safer than the current mixture of decoded and undecoded content. The second option suggests a piece of hardening that stands on its own. Now that a body buffer can be not-quite-plaintext, nothing downstream can tell. A buffer-level flag for "decompression incomplete, do not trust this as plaintext" would let rule authors and file extraction make an informed choice instead of a blind one. ### 7.3 Either way If partial decoding stays possible in any form, it deserves a signal that says what it is. What fires today reads as a request-side header complaint and says nothing about body coverage having been lost. Something along the lines of "decompression incomplete, body detection skipped for this transaction" would be actionable. What exists now is not. ## 8. Regression test `suricata-verify/http-content-encoding-multivalue/` next to this report contains `test.yaml`, `test.rules`, `input.pcap` and `README.md`. Its `input.pcap` is the same file as `evidence/f2-repro.pcap`. It asserts **4 alerts for 4 transactions**. On 8.0.7 it produces 3, so it is red before the fix and green after. I'd suggest landing it together with the fix rather than before, so your CI does not go red in the meantime. ## 9. Disclosure * Nothing here has been published. I searched the OISF/suricata tracker and the GitHub security advisories and found no existing report or advisory for this behaviour. * Credit is welcome under my GitHub handle, **Zer0Gate**. * The pcap and the payloads are non-confidential and may enter public CI or QA. * All of this was done on instances and infrastructure I own. --- ## Appendix A: files In `poc/suricata/round-03/ecs-verify/`: | file | what it does | |---|---| | `gen-f2.py` | generates the body variants and `manifest.json`; checks each one by decoding it back and comparing with `P` | | `serve.py` | test server; serves each variant with the matching `Content-Encoding`, and on request-direction runs decodes the received body and reports the result | | `collect-f2.py` | parses EVE into the §6.2 verdicts | | `make-pcap.py` | synthesises the reproducer: four complete HTTP/1.1 sessions, deterministic, no capture loss | | `payloads-f2/` | the generated bodies | In `evidence/`: | file | what it is | |---|---| | `f2-repro.pcap` | the reproducer, 102 packets | | `f2-repro.manifest.json` | sizes and hashes of the generated bodies | | `docker-repro.txt`, `docker-repro-eve.json`, `docker-repro-fast.log` | the §5.3 replay and its output | | `f2-verify-result.txt` | formatted verdicts from the §6.2 run | | `f2-verify-raw-eve.txt` | raw EVE from the §6.2 run | | `serve-request-log.txt` | endpoint decoding results, §6.3 | | `sv-run.txt` | the suricata-verify runner output from §6.5 | | `sv-output-cmdline.txt`, `sv-output-eve.json`, `sv-output-fast.log`, `sv-output-suricata.log` | what the runner's Suricata invocation produced | ## Appendix B: traps, in case they save you time Each of these produced a convincing but wrong conclusion while I was working, and each would do the same to anyone building a test on top of this bug. **B1. `zstd` proves nothing in this test.** zstd emits RAW_BLOCKs, so literal plaintext, canary included, ends up inside the zstd stream itself. A `Content-Encoding: zstd, gzip` body therefore matches a plaintext signature whether or not Suricata drops the layer, and its `fileinfo.size` stays near the compressed size either way. I first read that as "the bypass does not work on requests" and it was an artefact. I dropped both `zstd` variants from the table for this reason. Brotli does not have this property, which is why the `br` variants are the usable ones. Check: `gzip.decompress()` of the `zstd-gzip` body still contains the canary; the same on `br-gzip` and `br-deflate` does not. **B2. Direction matters as much as content.** ET Open's C2 signatures mostly read `alert http $EXTERNAL_NET any -> $HOME_NET any (... to_client ...)`, on the assumption that the client is inside and the C2 server is outside. In a test where the probed server is in `$HOME_NET` and the tester is outside, the response direction runs `$HOME_NET -> $EXTERNAL_NET` and those rules cannot fire. Easy to read as "evasion confirmed" when nothing was ever going to match. The response-direction canaries in §6.2 were picked because their direction fits. **B3. `flowbits`, `bsize` and `startswith` preconditions.** Several otherwise-simple body rules carry a `flowbits:isset,...` prerequisite that a synthetic test will never satisfy. Three of my first candidates had this problem. **B4. ET `sid:2049403` and `sid:2049404` cannot match on 8.0.7.** They combine `content:"127.0.0.1"; content:"localhost|0a|"; fast_pattern; within:10;` with `pcre:"/^127\.0\.0\.1\s+localhost[\r\n]+/m"`, which contradicts itself: `127.0.0.1` is 9 bytes and the pcre needs at least one whitespace byte before `localhost`, so `localhost\n` can never start before offset 10, while `within:10` limits the scan to `[0,10)` (`detect-engine-content-inspection.c:180-181`, `depth = prev_buffer_offset + within + distance`). `fast.log` on the instance confirms neither has ever fired. Unrelated to this report; it just quietly invalidated a control I had planned. **B5. A live capture can be truncated, and truncation looks like "no bug".** My first reproducer was a `tcpdump -i eth0 -s 0 -w ...` capture of exactly the traffic in §5.1. Every response body came out cut roughly in half: 32,646 / 4,752 / 4,442 / 4,445 bytes instead of 65,536 / 8,820 / 8,514 / 8,506. With bodies it could not reassemble, Suricata alerted on nothing, including the plaintext control, so that run appeared to prove the opposite. If your control does not fire either, suspect the capture before you suspect the finding. **B6. TCP flow control applies to hand-built pcaps too.** My first synthesised pcap advertised a 29,200-byte receive window and then had the server push a 65,753-byte response in one go. Suricata, correctly, refused to reassemble past the advertised window, so the plaintext control was truncated to 27,740 bytes while the three ~8.5 KB transactions were unaffected. That gives a partial evasion table that looks like an artefact of body size rather than a `Content-Encoding` problem. `make-pcap.py` now advertises 65,535 and inserts a client ACK between batches to reopen the window. ## Appendix C: what I am not claiming * No memory-safety impact. This is a detection-coverage defect. * TLS is out of scope. Like any IDS without decryption, encrypted bodies are not inspectable, and this report says nothing about that. * I did not test 7.0.x, and I make no claim about it. * I did not test IPS drop behaviour beyond noting that a rule which does not match cannot drop. The body is equally invisible in IDS and IPS mode.