Suricata

I have stumbled on a strange case that is reproducible only Bionic LTS (I could not reproduce it on latest Debian for example using same config/suricata/pcap/command line)

It seems the combination of the custom suricata.yaml plus the pcap and the stream event rules on Bionic - triggers the long wait on exit from reading a pcap - which seems strange.

Long wait on exit from pcap read

/opt/suricatagit/bin/suricata  -c fuzz.suricata.sandnet.socket.yaml  -k none -l log/ -r fc31ff29339e3d37180fbd6965ebe3ed.pcap -S /home/pmanev/Work/scripts/git-install/oisf-current/rules/stream-events.rules  

[693] 24/11/2018 -- 04:40:52 - (suricata.c:1085) <Notice> (LogVersion) -- This is Suricata version 4.1.0-dev (rev 683be948)
[693] 24/11/2018 -- 04:40:53 - (tm-threads.c:2172) <Notice> (TmThreadWaitOnThreadInit) -- all 33 packet processing threads, 2 management threads initialized, engine started.
[693] 24/11/2018 -- 04:40:54 - (suricata.c:2847) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[693] 24/11/2018 -- 04:41:55 - (tm-threads.c:1527) <Warning> (TmThreadDrainPacketThreads) -- [ERRCODE: SC_ERR_SHUTDOWN(188)] - unable to get all packet threads to process their packets in time
[694] 24/11/2018 -- 04:42:19 - (source-pcap-file.c:383) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 11843 packets, 10393386 bytes

Same Suricata , yaml config/pcap/ no stream rules loaded but with full ETPro - no issues

/opt/suricatagit/bin/suricata  -c fuzz.suricata.sandnet.socket.yaml  -k none -l log/ -r fc31ff29339e3d37180fbd6965ebe3ed.pcap -S "/opt/suricatagit/etc/etpro/ET*.rules" 

[592] 24/11/2018 -- 04:38:05 - (suricata.c:1085) <Notice> (LogVersion) -- This is Suricata version 4.1.0-dev (rev 683be948)
[592] 24/11/2018 -- 04:38:40 - (tm-threads.c:2172) <Notice> (TmThreadWaitOnThreadInit) -- all 33 packet processing threads, 2 management threads initialized, engine started.
[592] 24/11/2018 -- 04:38:40 - (suricata.c:2847) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[596] 24/11/2018 -- 04:38:40 - (source-pcap-file.c:383) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 11843 packets, 10393386 bytes

Same Suricata /pcap and stream rules loaded but with default config - no issues

/opt/suricatagit/bin/suricata  -k none -l log/ -r fc31ff29339e3d37180fbd6965ebe3ed.pcap -S /home/pmanev/Work/scripts/git-install/oisf-current/rules/stream-events.rules

[791] 24/11/2018 -- 04:43:38 - (suricata.c:1085) <Notice> (LogVersion) -- This is Suricata version 4.1.0-dev (rev 683be948)
[791] 24/11/2018 -- 04:43:39 - (tm-threads.c:2172) <Notice> (TmThreadWaitOnThreadInit) -- all 33 packet processing threads, 4 management threads initialized, engine started.
[791] 24/11/2018 -- 04:43:39 - (suricata.c:2847) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[792] 24/11/2018 -- 04:43:39 - (source-pcap-file.c:383) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 11843 packets, 10393386 bytes

/opt/suricatagit/bin/suricata --build-info
This is Suricata version 4.1.0-dev (rev 683be948)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON PROFILING TLS MAGIC RUST 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 7.3.0, C version 199901
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.28, linked against LibHTP v0.5.28

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  liblzma support:                         no
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  libgeoip:                                yes
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes

  Rust support:                            yes
  Rust strict mode:                        no
  Rust debug mode:                         no
  Rust compiler:                           rustc 1.28.0
  Rust cargo:                              cargo 1.28.0

  Suricatasc install:                      yes

  Profiling enabled:                       yes
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     yes
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /opt/suricatagit
  Configuration directory:                 /opt/suricatagit/etc/suricata/
  Log directory:                           /opt/suricatagit/var/log/suricata/

  --prefix                                 /opt/suricatagit
  --sysconfdir                             /opt/suricatagit/etc
  --localstatedir                          /opt/suricatagit/var

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native -I${srcdir}/../rust/gen/c-headers
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                                

ldd /opt/suricatagit/bin/suricata
    linux-vdso.so.1 (0x00007ffcc4368000)
    libhtp.so.2 => /opt/suricatagit/lib/libhtp.so.2 (0x00007f58217fe000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f58215fa000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f58213f2000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5821054000)
    liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f5820e38000)
    libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 (0x00007f5820c04000)
    libluajit-5.1.so.2 => /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2 (0x00007f582098b000)
    libmagic.so.1 => /usr/lib/x86_64-linux-gnu/libmagic.so.1 (0x00007f5820769000)
    libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007f5820564000)
    libpcap.so.0.8 => /usr/lib/x86_64-linux-gnu/libpcap.so.0.8 (0x00007f5820323000)
    libnet.so.1 => /usr/lib/x86_64-linux-gnu/libnet.so.1 (0x00007f5820109000)
    libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f581fefb000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f581fcdc000)
    libyaml-0.so.2 => /usr/lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007f581fabe000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f581f84c000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f581f62f000)
    libhs.so.4 => /usr/lib/x86_64-linux-gnu/libhs.so.4 (0x00007f581eb2a000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f581e7e6000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f581e5a9000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f581e391000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f581dfa0000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f582231e000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f581dc12000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f581d9e3000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f581d7de000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f581d5da000)

pcap and configs - privately shared.