Actions
Bug #7497
closedpcap: exit with errors when running with -r and --pcap-file-continuous
Description
Hi,
I'm trying to run suricata as follows:
suricata -r /tmp/pcaps/ --pcap-file-continuous --pcap-file-delete -vvvv
It shows the following logs and then exits:
Info: pcap: Processing pcaps directory /tmp/pcaps/, files must be newer than 0 and older than 1705604840362 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497]
Info: pcap: Processing pcaps directory /tmp/pcaps/, files must be newer than 0 and older than 1705604845362 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497]
Info: pcap: Processing pcaps directory /tmp/pcaps/, files must be newer than 0 and older than 1705604850363 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497]
Error: threads: thread "FM#01" failed to start in time: flags 0003 [TmThreadWaitOnThreadRunning:tm-threads.c:1832]
This issue was already addressed here - https://forum.suricata.io/t/suricata-exits-with-errors-when-running-with-r-and-pcap-file-continuous/4396.
However, it seems that it wasn't fixed (I've tested in both 7.0.2 and 7.0.8).
The suggested solution to use suricatasc is not good for me for two reasons:
1. It's problematic for me - https://redmine.openinfosecfoundation.org/issues/7283.
2. It seems there is another bug that once running the command it disables the
pcap-file.delete-when-doneThanks ahead
Updated by Victor Julien 6 months ago
- Related to Bug #7568: pcap: continuous file reading fails on an empty directory added
Updated by Victor Julien 6 months ago
- Subject changed from Suricata exits with errors when running with -r and –pcap-file-continuous to pcap: exit with errors when running with -r and --pcap-file-continuous
Updated by Philippe Antoine about 1 month ago
- Status changed from New to Feedback
I am not reproducing with 8, was this not fixed by #7568 ?
Updated by Philippe Antoine about 1 month ago
- Target version changed from TBD to 7.0.12
- Affected Versions 7.0.11 added
Reproducing with 7.0.11 but not with 8
Updated by Philippe Antoine about 1 month ago
- Assignee changed from OISF Dev to Lukas Sismis
Lukas, would you know better what is happening ?
(On my device, I have to wait a few minutes before getting the failure)
Updated by Lukas Sismis 23 days ago
Hi @Philippe Antoine,
I tried it on 7.0.11 compiled from source, it does not crash on my side, and I've waited for over 10 minutes. So I would consider this solved by #7568
Can you please re-verify?
Thanks.
Updated by Philippe Antoine 22 days ago · Edited
I re-verified and I confirm the bug still exists in main-7.0.x, with pcaps being an empty directory
My logs
./src/suricata -c suricata.yaml -r pcaps/ --pcap-file-continuous --pcap-file-delete -vvvv Notice: suricata: This is Suricata version 7.0.11-dev (f3b544eec8 2025-06-13) running in USER mode [LogVersion:suricata.c:1159] Info: cpu: CPUs/cores online: 16 [UtilCpuPrintSummary:util-cpu.c:182] Info: suricata: Setting engine mode to IDS mode by default [PostConfLoadedSetup:suricata.c:2699] Info: exception-policy: master exception-policy set to: auto [ExceptionPolicyMasterParse:util-exception-policy.c:200] Config: exception-policy: app-layer.error-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: app-layer-htp: 'default' server has 'request-body-minimal-inspect-size' set to 33023 and 'request-body-inspect-window' set to 4035 after randomization. [HTPConfigSetDefaultsPhase2:app-layer-htp.c:2589] Config: app-layer-htp: 'default' server has 'response-body-minimal-inspect-size' set to 42821 and 'response-body-inspect-window' set to 16035 after randomization. [HTPConfigSetDefaultsPhase2:app-layer-htp.c:2602] Config: smb: read: max record size: 16777216, max queued chunks 64, max queued size 67108864 [suricata::smb::smb::rs_smb_register_parser:smb.rs:2439] Config: smb: write: max record size: 16777216, max queued chunks 64, max queued size 67108864 [suricata::smb::smb::rs_smb_register_parser:smb.rs:2441] Config: app-layer-enip: Protocol detection and parser disabled for enip protocol. [RegisterENIPUDPParsers:app-layer-enip.c:538] Config: app-layer-dnp3: Protocol detection and parser disabled for DNP3. [RegisterDNP3Parsers:app-layer-dnp3.c:1575] Config: host: allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64 [HostInitConfig:host.c:256] Config: host: preallocated 1000 hosts of size 136 [HostInitConfig:host.c:282] Config: host: host memory usage: 398144 bytes, maximum: 33554432 [HostInitConfig:host.c:284] Config: coredump-config: Core dump size set to unlimited. [CoredumpLoadConfig:util-coredump-config.c:155] Config: exception-policy: defrag.memcap-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: defrag-hash: allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56 [DefragInitConfig:defrag-hash.c:251] Config: defrag-hash: preallocated 65535 defrag trackers of size 160 [DefragInitConfig:defrag-hash.c:280] Config: defrag-hash: defrag memory usage: 14155616 bytes, maximum: 33554432 [DefragInitConfig:defrag-hash.c:287] Config: exception-policy: flow.memcap-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: flow: flow size 296, memcap allows for 453438 flows. Per hash row in perfect conditions 6 [FlowInitConfig:flow.c:681] Config: stream-tcp: stream "prealloc-sessions": 2048 (per thread) [StreamTcpInitConfig:stream-tcp.c:418] Config: stream-tcp: stream "memcap": 67108864 [StreamTcpInitConfig:stream-tcp.c:438] Config: stream-tcp: stream "midstream" session pickups: disabled [StreamTcpInitConfig:stream-tcp.c:446] Config: stream-tcp: stream "async-oneside": disabled [StreamTcpInitConfig:stream-tcp.c:454] Config: stream-tcp: stream "checksum-validation": enabled [StreamTcpInitConfig:stream-tcp.c:469] Config: exception-policy: stream.memcap-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: exception-policy: stream.reassembly.memcap-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: exception-policy: stream.midstream-policy: ignore (defined via 'exception-policy' master switch) [ExceptionPolicyGetDefault:util-exception-policy.c:219] Config: stream-tcp: stream."inline": disabled [StreamTcpInitConfig:stream-tcp.c:501] Config: stream-tcp: stream "bypass": disabled [StreamTcpInitConfig:stream-tcp.c:514] Config: stream-tcp: stream.reassembly.urgent.policy": inline [StreamTcpInitConfig:stream-tcp.c:545] Config: stream-tcp: stream "max-syn-queued": 10 [StreamTcpInitConfig:stream-tcp.c:581] Config: stream-tcp: stream "max-synack-queued": 5 [StreamTcpInitConfig:stream-tcp.c:594] Config: stream-tcp: stream.reassembly "memcap": 268435456 [StreamTcpInitConfig:stream-tcp.c:615] Config: stream-tcp: stream.reassembly "depth": 1048576 [StreamTcpInitConfig:stream-tcp.c:634] Config: stream-tcp: stream.reassembly "toserver-chunk-size": 2453 [StreamTcpInitConfig:stream-tcp.c:706] Config: stream-tcp: stream.reassembly "toclient-chunk-size": 2550 [StreamTcpInitConfig:stream-tcp.c:708] Config: stream-tcp: stream.reassembly.raw: enabled [StreamTcpInitConfig:stream-tcp.c:721] Config: stream-tcp: stream.liberal-timestamps: disabled [StreamTcpInitConfig:stream-tcp.c:730] Config: stream-tcp-reassemble: stream.reassembly "segment-prealloc": 2048 [StreamTcpReassemblyConfig:stream-tcp-reassemble.c:493] Config: stream-tcp-reassemble: stream.reassembly "max-regions": 8 [StreamTcpReassemblyConfig:stream-tcp-reassemble.c:516] Info: logopenfile: fast output device (regular) initialized: fast.log [SCConfLogOpenGeneric:util-logopenfile.c:620] Info: logopenfile: eve-log output device (regular) initialized: eve.json [SCConfLogOpenGeneric:util-logopenfile.c:620] Config: runmodes: enabling 'eve-log' module 'alert' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'frame' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'anomaly' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'http' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'dns' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'tls' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'files' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'smtp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'ftp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'rdp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'nfs' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'smb' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'tftp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'ike' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'dcerpc' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'krb5' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'bittorrent-dht' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'snmp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'rfb' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'sip' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'quic' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'dhcp' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'ssh' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'mqtt' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'http2' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'pgsql' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'stats' [RunModeInitializeEveOutput:runmodes.c:715] Config: runmodes: enabling 'eve-log' module 'flow' [RunModeInitializeEveOutput:runmodes.c:715] Info: logopenfile: stats output device (regular) initialized: stats.log [SCConfLogOpenGeneric:util-logopenfile.c:620] Config: suricata: Delayed detect disabled [SetupDelayedDetect:suricata.c:2408] Config: detect: pattern matchers: MPM: hs, SPM: hs [DetectEngineCtxInitReal:detect-engine.c:2515] Config: detect: grouping: tcp-whitelist (default) 53, 80, 139, 443, 445, 1433, 3306, 3389, 6666, 6667, 8080 [DetectEngineCtxLoadConf:detect-engine.c:2938] Config: detect: grouping: udp-whitelist (default) 53, 135, 5060 [DetectEngineCtxLoadConf:detect-engine.c:2964] Config: detect: prefilter engines: MPM [DetectEngineCtxLoadConf:detect-engine.c:2994] Config: reputation: IP reputation disabled [SRepInit:reputation.c:606] Warning: classification-config: could not open: "/usr/local/etc/suricata/classification.config": No such file or directory [SCClassConfInitContextAndLocalResources:util-classification-config.c:135] Warning: detect: No rule files match the pattern /usr/local/var/lib/suricata/rules/suricata.rules [ProcessSigFiles:detect-engine-loader.c:240] Config: detect: No rules loaded from suricata.rules. [SigLoadSignatures:detect-engine-loader.c:330] Warning: detect: 1 rule files specified, but no rules were loaded! [SigLoadSignatures:detect-engine-loader.c:355] Warning: threshold-config: Error opening file: "/usr/local/etc/suricata//threshold.config": No such file or directory [SCThresholdConfInitContext:util-threshold-config.c:178] Info: detect: 0 signatures processed. 0 are IP-only rules, 0 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only [SigAddressPrepareStage1:detect-engine-build.c:1880] Config: detect: building signature grouping structure, stage 1: preprocessing rules... complete [SigAddressPrepareStage1:detect-engine-build.c:1886] Perf: detect: TCP toserver: 0 port groups, 0 unique SGH's, 0 copies [RulesGroupByPorts:detect-engine-build.c:1665] Perf: detect: TCP toclient: 0 port groups, 0 unique SGH's, 0 copies [RulesGroupByPorts:detect-engine-build.c:1665] Perf: detect: UDP toserver: 0 port groups, 0 unique SGH's, 0 copies [RulesGroupByPorts:detect-engine-build.c:1665] Perf: detect: UDP toclient: 0 port groups, 0 unique SGH's, 0 copies [RulesGroupByPorts:detect-engine-build.c:1665] Perf: detect: OTHER toserver: 0 proto groups, 0 unique SGH's, 0 copies [RulesGroupByProto:detect-engine-build.c:1055] Perf: detect: OTHER toclient: 0 proto groups, 0 unique SGH's, 0 copies [RulesGroupByProto:detect-engine-build.c:1088] Perf: detect: Unique rule groups: 0 [SigAddressPrepareStage4:detect-engine-build.c:2066] Perf: detect: Builtin MPM "toserver TCP packet": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "toclient TCP packet": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "toserver TCP stream": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "toclient TCP stream": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "toserver UDP packet": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "toclient UDP packet": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Perf: detect: Builtin MPM "other IP packet": 0 [MpmStoreReportStats:detect-engine-mpm.c:1468] Config: tmqh-flow: AutoFP mode using "Hash" flow load balancer [TmqhFlowPrintAutofpHandler:tmqh-flow.c:92] Info: pcap: Argument pcaps/ was a directory [ReceivePcapFileThreadInit:source-pcap-file.c:281] Config: flow-manager: using 1 flow manager threads [FlowManagerThreadSpawn:flow-manager.c:992] Config: flow-manager: using 1 flow recycler threads [FlowRecyclerThreadSpawn:flow-manager.c:1198] Info: pcap: Starting directory run for pcaps/ [ReceivePcapFileLoop:source-pcap-file.c:183] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814283985 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814283985 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814288985 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814293985 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814298985 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814303986 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814308986 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814313986 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814318986 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814323987 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814328987 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814333987 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Info: pcap: Processing pcaps directory pcaps/, files must be newer than 0 and older than 1753814338987 [PcapDirectoryDispatch:source-pcap-file-directory-helper.c:497] Error: threads: thread "FM#01" failed to start in time: flags 0003. Total threads: 4. Time budget 64s [WaitOnThreadsRunningByType:tm-threads.c:1840]
Updated by Philippe Antoine 22 days ago
- Status changed from Feedback to Closed
- Target version changed from 7.0.12 to 7.0.11
My bad, my git pull did not work, this is fixed indeed in 7.0.11
Actions