Suricata

# D8 — byte_extract values silently truncated to 32 bits at their consumer sites

**Status:** REPRODUCED on 8.0.7 RELEASE (branch HEAD `608ea09ad`), normal and ASAN builds.

## TL;DR

`byte_extract` (and `byte_math`) store their results as `uint64_t`. Every keyword that
*consumes* such a variable (`content` `distance`/`within`/`offset`/`depth`, `byte_test`,
`byte_jump`) casts it to a 32-bit integer before use. The upper bits are dropped and, for
`distance`, values ≥ 2³¹ additionally turn negative. As a result, a rule can match content at a
position the extracted value says is unreachable.

Example: a length field of `0x1_0000_0000` (4 GiB) is interpreted as `distance:0`, so
content immediately following the field matches, although by the rule's semantics it is
4 GiB away.

The code already acknowledges the problem: `src/detect-engine-content-inspection.c` has 9
comments reading *"This cast is wrong if a 64-bit value was extracted"*.

## Can a >32-bit value be extracted at all?

Yes. The user guide (`doc/userguide/rules/payload-keywords.rst`, `byte_extract` section)
states no maximum for `<num of bytes>`, but the code accepts:

| Extraction mode | Max `nbytes` | Source |
|---|---|---|
| binary (no `string`) | 8 | `NO_STRING_MAX_BYTES_TO_EXTRACT`, `src/detect-byte-extract.c:58,224` |
| `string,hex` | 14 | `src/detect-byte-extract.c:56` |
| `string,dec` | 20 | `src/detect-byte-extract.c:55` |
| `string,oct` | 23 | `src/detect-byte-extract.c:54` |

The value is read with `ByteExtractUint64()` (`src/detect-byte-extract.c:144`) into
`det_ctx->byte_values[]`, which is declared `uint64_t *` (`src/detect.h:1328`).

The 1–4 byte limit that many rule writers expect comes from Snort. It does not apply here.
Even with ≤ 4 bytes extracted, the stored value can still exceed 32 bits:

- `multiplier` accepts values up to 65535 (`rust/src/detect/byte_extract.rs:208`) and is applied
  in u64: `val *= data->multiplier_value` (`src/detect-byte-extract.c:153`).
- `string,dec` of `"4294967296"` (10 ASCII bytes) is 2³².
- `byte_math` variables (`bytes` 1–10, `rust/src/detect/byte_math.rs:317`) are computed in u64
  and feed the same consumers.

Separately, a plain 4-byte value ≥ `0x80000000` becomes a **negative** `distance`, because the
result of the `(uint32_t)` cast is stored in an `int`.

## Where the truncation happens

All sites are in `src/detect-engine-content-inspection.c`:

| Line | Keyword / modifier | Cast | Effect of truncation |
|---|---|---|---|
| 161 | content `distance:var` | `(uint32_t)` → `int` | window starts too early; ≥ 2³¹ → negative distance (**tested**) |
| 175–177 | content `within:var` | `(int32_t)` compare, `(uint32_t)` assign | window may shrink (by inspection) |
| 215–217 | content `offset:var` (relative) | `(uint32_t)` | guard compares in u64, then assigns truncated value (by inspection) |
| 229 | content `depth:var` (absolute) | `(uint32_t)` | depth may shrink (by inspection) |
| 249 | content `offset:var` (absolute) | `(uint32_t)` | offset may shrink (by inspection) |
| 512, 519 | `byte_test` offset / nbytes var | `(int32_t)` | by inspection |
| 546, 552 | `byte_jump` offset / nbytes var | `(int32_t)` | by inspection |

Line 200–202 (relative `depth:var`) is **not** affected: it compares in u64 before casting,
and its comment says so.

## Reproduction

### Rule shape and how it evaluates

```
byte_extract:8,0,var1,little; content:"<PAT>"; distance:var1;
```

1. `byte_extract:8,0,…` reads 8 bytes at offset 0 as a little-endian u64 into `var1` and
   moves the relative cursor to offset 8 (end of the extracted bytes).
2. `distance:var1` requires `<PAT>` to start at least `var1` bytes after that cursor. The search
   window therefore starts at `8 + var1`.

Each test packet is `<8-byte LE value><filler><pattern>`. Each rule looks for its own pattern,
so it can only match its own packet:

| sid | var1 in packet | bytes after offset 8 | pattern at | correct window start | correct result | engine today (truncated var) |
|---|---|---|---|---|---|---|
| 1 | 4 | `XXXX AAAA` | 12 | 8+4 = 12 | **match** (12 ≥ 12) | 4 → match ✓ |
| 2 | 2³² | `BBBB` | 8 | 8+2³² | **no match** (beyond any buffer) | 0 → window at 8 → match ✗ |
| 3 | 2³²+4 | `XXXX CCCC` | 12 | 8+2³²+4 | **no match** (beyond any buffer) | 4 → window at 12 → match ✗ |
| 4 | 5 | `XXXX DDDD` | 12 | 8+5 = 13 | **no match** (12 < 13) | 5 → no match ✓ |

- **sid 1 (positive control)** shows that the rule shape, the pcap and the engine work.
- **sids 2 and 3** use the same layout as sid 1, but the distance has bit 32 set. They fire
  only because the engine does not see that bit. Each pattern sits exactly where the truncated
  value points, so whether the rule fires tells you directly whether the upper bits were
  dropped.
- **sid 4 (negative control)** shows that a distance just one byte too large still correctly
  blocks the match. So sids 2 and 3 fire because of the truncation, not because `distance`
  is ignored.

### Expected vs. observed

| sid | Correct behaviour | Observed on 8.0.7 |
|---|---|---|
| 1 | 1 alert | 1 alert |
| 2 | **0 alerts** | 1 alert ✗ |
| 3 | **0 alerts** | 1 alert ✗ |
| 4 | 0 alerts | 0 alerts |

Normal and ASAN builds behave identically. There is no memory-safety error; this is purely a
detection-logic bug.

### Additional variants (verified ad hoc, not yet in `reproducer.sh`)

Both variants alerted on 8.0.7, and neither needs an 8-byte extraction:

| Rule | Payload | Why it must not match | Result |
|---|---|---|---|
| `byte_extract:4,0,v,multiplier 256; content:"MMMM"; distance:v;` | `01 00 00 00` + `MMMM` | 2²⁴·256 = 2³² → 0 | alert ✗ |
| `byte_extract:10,0,v,string,dec; content:"SSSS"; distance:v;` | `"4294967296"` + `SSSS` | 2³² → 0 | alert ✗ |
| `byte_extract:4,4,v; content:"NNNN"; distance:v;` | `NNNN` + `FF FF FF F8` | 0xFFFFFFF8 → `int` −8 → matches *before* the extracted field | alert ✗ |

I did not find an existing Redmine issue, so creating this one to cover it.