Bug #8603
openpcap-file mode keeps threshold state across pcaps
Description
In Suricata 8.0.4, Unix socket pcap-file mode appears to preserve rule threshold state across separately submitted pcaps. This causes alerts in one pcap to suppress alerts in a later pcap for rules using thresholding, for example:
threshold: type limit, count 1, seconds 30, track by_src;
I am processing many single pcap files, submitted one at a time through the Unix socket. I expect threshold state to reset per pcap file, or at least to have a config option to enable that behavior.
Reproduction¶
- Start Suricata with Unix socket pcap processing.
suricata -c suricata.yaml --unix-socket
- Load a rule that matches traffic in both pcaps and includes:
threshold: type limit, count 1, seconds 30, track by_src;
- Prepare two pcaps:
pcap1contains matching traffic from source IPA.pcap2contains matching traffic from the same source IPA.- The matching packet timestamp in
pcap2is within 30 seconds of the matching packet timestamp inpcap1.
- Submit them as separate single-file jobs:
pcap-file /path/pcap1.pcap /tmp/out1 pcap-file /path/pcap2.pcap /tmp/out2
Expected behavior¶
Each submitted pcap should be processed independently when desired. With per-pcap threshold reset enabled, pcap2 should emit its own alert even if pcap1 triggered the same thresholded rule within the threshold window.
Actual behavior¶
pcap1 emits the alert. pcap2 does not emit the alert because the threshold entry from pcap1 remains active.
Relevant code paths¶
Unix socket pcap processing handles queued files in src/runmode-unix-socket.c, UnixSocketPcapFilesCheck.
Before the next file starts, it logs "Resetting engine state" and calls:
PostRunDeinit(RUNMODE_PCAP_FILE, NULL);
This is in src/runmode-unix-socket.c around line 453. The next pcap is then started via PreRunInit, PreRunPostPrivsDropInit, and RunModeDispatch around lines 545-547.
PostRunDeinit in src/suricata.c cleans packet/runtime state such as packet pool, flow, ippair, host, stream, defrag, and HTTP range containers, but it does not clear threshold state.
Threshold state is held globally in src/detect-engine-threshold.c:
static struct Thresholds {
THashTableContext *thash;
} ctx;
This is initialized through ThresholdInit() during Suricata startup in src/suricata.c and destroyed only during global shutdown. It is not reset between Unix socket pcap jobs.
For track by_src, threshold lookup uses a key including sid, gid, rev, track, tenant_id, and the source address in src/detect-engine-threshold.c, around ThresholdGetFromHash. It does not include the pcap file or pcap job identity.
For type limit, the code suppresses additional matches while the packet timestamp remains inside the threshold window. Pcap packet timestamps come from the pcap itself in src/source-pcap-file-helper.c.
Tenant mode consideration¶
Tenant IDs do not seem like a good workaround for this.
The Unix socket pcap-file command accepts an optional tenant argument, and the threshold key includes tenant_id. However, tenant IDs are intended for Suricata multi-tenancy, where different tenants can use different rule/config contexts.
Using tenant IDs only to isolate pcap jobs would require enabling/configuring multi-detect with direct selection and likely duplicating otherwise identical tenant configurations. That is operationally heavy and changes the detection/logging model. In normal single-tenant pcap scanning, packets use tenant id 0, meaning no tenant is assigned.
Possible fix¶
Add a pcap-file option such as:
pcap-file: reset-thresholds: false
Defaulting to false would preserve current behavior. When set to true, Unix socket pcap mode would clear the rule threshold table between submitted pcap files, likely after PostRunDeinit(RUNMODE_PCAP_FILE, ...) and before starting the next queued pcap.
My expected usage would be:
pcap-file: reset-thresholds: true
I am not sure how expensive this reset would be. It would happen only between pcap files in this mode, so it may be acceptable, but maintainers may have a better view of the performance tradeoff.
I can provide an implementation once the maintainers confirm that this behavior and configuration approach are acceptable.