Suricata

We also could implement this by making something like pass_prior_hooks be a keyword that customers can add to their rules - either way should work but just wanted to throw an additional idea in case it may make more sense to you all

Design Proposal¶

One author-visible rule. The Rule_Loader auto-synthesises the accept chain from the protocol's registered state machine. Two equivalent syntaxes:

accept:tx <tls:client_hello_done $HOME_NET any -> $EXTERNAL_NET any \
    (tls.sni; content:".amazon.com"; endswith; sid:1003;)

or, using a keyword form:

accept:tx tls:client_hello_done $HOME_NET any -> $EXTERNAL_NET any \
    (tls.sni; content:".amazon.com"; endswith; accept-prior-states; sid:1003;)

Both forms produce the same expansion. All protocol knowledge (state ordinals, directions, registered transports, keyword→state mappings) comes from the existing AppLayerParser* and DetectBufferType registries — no per-protocol code in the loader. Multi-transport protocols are handled by the expander iterating every transport the registry reports for the protocol, so a single DNS rule emits both the TCP handshake block and the UDP transport block.

Design decision 1 — Load-time pre-processor expansion (chosen) vs lazy evaluation¶

The Rule_Loader expands each Prior_State_Rule into concrete accept-rule strings before DetectFirewallRuleAppendNew sees them. Runtime sees ordinary Signature objects.

Pros (chosen approach).

• Zero per-packet cost: equivalent hand-written rules = identical hot path.
• Round-trip equivalence with hand-written rules is structural, not behavioural — same code path from parsing onward.
• No cross-table coupling: the TCP handshake rules must live in packet_filter and the state rules in app_filter; load-time expansion handles this naturally.
• Pure loader-side change, no reach into detect.c / alert / tagging — easier upstream review surface.

Cons.

• Engine-internal rule count grows (10-13× per Prior_State_Rule). Invisible to customers but real in MPM compilation budget. Measured in microseconds per rule for assembly; MPM compilation dominates the load-time budget either way.

Alternative considered — native lazy evaluation. Keep the Prior_State_Rule as a single Signature with a cached prerequisite bitmap, and check "does this packet's current state satisfy the bitmap?" lazily on every detection pass.

Pros (alternative).

• No rule-count explosion in the detection engine.
• Expansion is a runtime concept, not a load-time artefact.

Cons (alternative — why rejected).

• Permanent per-packet branch on every Signature evaluation (is this Prior_State? does current state satisfy?). Adds non-zero cost to the hot path.
• Round-trip equivalence becomes a behavioural claim that two different evaluators have to keep producing identical verdicts across every future engine change.
• packet_filter ↔ app_filter coupling for the TCP handshake side either requires an O(N-rules) per-packet scan or a load-time shadow into packet_filter — which is effectively pre-processor expansion at a different layer.
• Touches detect.c, detect-engine-alert.c, possibly detect-engine-tag.c — larger patch, more upstream review, longer path to merge.

Net: lazy evaluation is workable but pays a permanent runtime cost and a recurring correctness burden. Pre-processor expansion costs nothing at runtime and is a single loader-side change.

Design decision 2 — Customer-facing SID abstraction¶

Every customer-visible record (eve.alert, fast.log, syslog, any SID-keyed tooling) shows only the author's Parent_SID. Derived Sub_SIDs exist, but only as engineer-facing attribution in --dump-expanded-rules output and --engine-analysis JSON.

Two structural pieces enforce this:

1. Every loaded Signature keeps a unique uint32_t runtime SID. Suricata's existing uniqueness invariant (detect-parse.c:DetectEngineSignatureIsDuplicate, relied on by thresholding / tagging / dedup) stays intact. Auto-accepted rules get derived runtime SIDs under the deterministic formula 0x80000000 | (fnv1a32(file) ^ parent_sid ^ sub_index) & 0x7FFFFFFF.
2. Every auto-accepted Expanded_Rule carries noalert;. Today accept:* is already silent at the alert layer (PacketAlertFinalize skips (action & (ACTION_ALERT | ACTION_PASS)) == 0), but noalert; makes the silence a durable contract instead of a default that could change upstream.

The Decision_Hook rule is not marked noalert; — it is the single rule carrying the Parent_SID and is the only place customer-facing output from the expansion is allowed to originate.

Alternative considered — expose Sub_SIDs via optional eve metadata (firewall.log-expanded-from: yes, metadata.expanded_from: { parent_sid, sub_index } on each alert record).

Pros. Operators debugging a dropped flow can tell which sub-rule matched without --dump-expanded-rules.

Cons (why rejected). Leaks an implementation detail to a customer-facing surface the feature otherwise keeps invisible. Every operator tool keyed on sid would have to decide whether to group on sid or on metadata.expanded_from.parent_sid. Rejected; kept as engineer-facing only.

--dump-expanded-rules — testing tool, not a hard requirement¶

The spec defines a RUNMODE_DUMP_EXPANDED_RULES run mode that loads the ruleset (classifier → parser → validator → expander) and dumps the post-expansion concrete rules to stdout. It's an engineer-facing inspection tool with three uses:

• Demo-able visibility: the 12-rule TLS expansion and 10-rule DNS expansion are observable on stdout before any packets are processed, so reviewers can see what the pre-processor produces.
• Golden-file CI fixture: the Suricata-verify test fw-prior-state-dump-expanded diffs the dump against a checked-in dump.expected so rule-shape regressions surface on every build.
• Debugging handle for authors: paste the output back into a rule file to verify round-trip equivalence with hand-written rules.

This is not a hard user-facing requirement — no customer needs it to use the feature. It's scaffolding that makes the pre-processor approach auditable and keeps the engineer-inspection deliverables (--engine-analysis integration) cheap. Based on the preference of a different engineer-tooling surface, the expansion work stands on its own and the dump run mode can be dropped or replaced.