Suricata

There is descriptive enough message of what potentially could be the problem - which in this case it is:

stream.reassembly.segment-prealloc = 2000048

is too big for the stream.reassembly.memcap. However , wondering if such a calculation should not be made prior and just fail to start?

[245610] 23/8/2020 -- 12:06:33 - (stream-tcp-reassemble.c:467) <Error> (StreamTcpReassembleInitThreadCtx) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - failed to setup/expand stream segment pool. Expand stream.reassembly.memcap?
AddressSanitizer:DEADLYSIGNAL
=================================================================
==245498==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5630cf6ec3f9 bp 0x7f9ddc06d660 sp 0x7f9ddc06d640 T4)
==245498==The signal is caused by a READ memory access.
==245498==Hint: address points to the zero page.
    #0 0x5630cf6ec3f8 in StreamTcpReassembleFreeThreadCtx /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/stream-tcp-reassemble.c:478
    #1 0x5630cf6d5a31 in StreamTcpThreadDeinit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/stream-tcp.c:5280
    #2 0x5630cf5b340c in FlowWorkerThreadDeinit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/flow-worker.c:307
    #3 0x5630cf5b2f63 in FlowWorkerThreadInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/flow-worker.c:266
    #4 0x5630cf7116c0 in TmThreadsSlotVar /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/tm-threads.c:394
    #5 0x7f9deb171ea6 in start_thread nptl/pthread_create.c:477
    #6 0x7f9dea516eae in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfdeae)

Full info bellow:

pevma@~/inthetrenches/Suricata/suricomp$ /opt/suritest/bin/suricata  -S "rules/*.rules" -l logs/ -k none -r any.pcap  

[245498] 23/8/2020 -- 12:03:11 - (suricata.c:1065) <Notice> (LogVersion) -- This is Suricata version 6.0.0-dev (ac491c6e8 2020-08-07) running in USER mode
[245498] 23/8/2020 -- 12:03:12 - (flow.c:635) <Notice> (FlowInitConfig) -- flow size 328, memcap allows for 409200 flows. Per hash row in perfect conditions 6
[245498] 23/8/2020 -- 12:04:09 - (detect-flowbits.c:588) <Warning> (DetectFlowbitsAnalyze) -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Terse.Pastebin' is checked but not set. Checked in 2813075 and 1 other sigs
[245498] 23/8/2020 -- 12:04:09 - (detect-flowbits.c:588) <Warning> (DetectFlowbitsAnalyze) -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.DocVBAProject' is checked but not set. Checked in 2020170 and 0 other sigs
[245498] 23/8/2020 -- 12:04:09 - (detect-flowbits.c:588) <Warning> (DetectFlowbitsAnalyze) -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ETPROtxtminhead' is checked but not set. Checked in 2843620 and 3 other sigs
[245498] 23/8/2020 -- 12:04:09 - (detect-flowbits.c:588) <Warning> (DetectFlowbitsAnalyze) -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.telnet.busybox' is checked but not set. Checked in 2023019 and 2 other sigs
[245610] 23/8/2020 -- 12:06:33 - (util-pool.c:168) <Error> (PoolInit) -- [ERRCODE: SC_ERR_POOL_INIT(66)] - alloc error
[245610] 23/8/2020 -- 12:06:33 - (util-pool-thread.c:136) <Error> (PoolThreadExpand) -- [ERRCODE: SC_ERR_POOL_INIT(66)] - pool grow failed
[245610] 23/8/2020 -- 12:06:33 - (stream-tcp-reassemble.c:467) <Error> (StreamTcpReassembleInitThreadCtx) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - failed to setup/expand stream segment pool. Expand stream.reassembly.memcap?
AddressSanitizer:DEADLYSIGNAL
=================================================================
==245498==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5630cf6ec3f9 bp 0x7f9ddc06d660 sp 0x7f9ddc06d640 T4)
==245498==The signal is caused by a READ memory access.
==245498==Hint: address points to the zero page.
    #0 0x5630cf6ec3f8 in StreamTcpReassembleFreeThreadCtx /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/stream-tcp-reassemble.c:478
    #1 0x5630cf6d5a31 in StreamTcpThreadDeinit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/stream-tcp.c:5280
    #2 0x5630cf5b340c in FlowWorkerThreadDeinit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/flow-worker.c:307
    #3 0x5630cf5b2f63 in FlowWorkerThreadInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/flow-worker.c:266
    #4 0x5630cf7116c0 in TmThreadsSlotVar /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/tm-threads.c:394
    #5 0x7f9deb171ea6 in start_thread nptl/pthread_create.c:477
    #6 0x7f9dea516eae in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfdeae)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/stream-tcp-reassemble.c:478 in StreamTcpReassembleFreeThreadCtx
Thread T4 (W#03) created by T0 (Suricata-Main) here:
    #0 0x7f9deba679b2 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x399b2)
    #1 0x5630cf717a92 in TmThreadSpawn /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/tm-threads.c:1721
    #2 0x5630cf66f650 in RunModeFilePcapAutoFp /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/runmode-pcap-file.c:227
    #3 0x5630cf677a0a in RunModeDispatch /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/runmodes.c:391
    #4 0x5630cf707338 in SuricataMain /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:2805
    #5 0x5630cf31cad3 in main /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/main.c:22
    #6 0x7f9dea43fcc9 in __libc_start_main ../csu/libc-start.c:308

==245498==ABORTING

real    3m23.002s
user    3m13.314s
sys    0m8.912s
pevma@~/inthetrenches/Suricata/suricomp$ 
pevma@~/inthetrenches/Suricata/suricomp$ /opt/suritest/bin/suricata --dump-config |grep preall
defrag.prealloc = yes
flow.prealloc = 10000
stream.reassembly.segment-prealloc = 2000048
host.prealloc = 1000

pevma@~/inthetrenches/Suricata/suricomp$ /opt/suritest/bin/suricata --build-info
This is Suricata version 6.0.0-dev (ac491c6e8 2020-08-07)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS TLS_C11 MAGIC RUST 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 9.3.0, C version 201112
compiled with -fstack-protector-all
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.33, linked against LibHTP v0.5.33

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  GeoIP2 support:                          yes
  Non-bundled htp:                         no
  Old barnyard2 support:                   
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes

  Rust support:                            yes
  Rust strict mode:                        yes
  Rust compiler path:                      /home/pevma/.cargo/bin/rustc
  Rust compiler version:                   rustc 1.45.2 (d3fb005a3 2020-07-31)
  Cargo path:                              /home/pevma/.cargo/bin/cargo
  Cargo version:                           cargo 1.45.1 (f242df6ed 2020-07-22)
  Cargo vendor:                            yes

  Python support:                          yes
  Python path:                             /usr/bin/python3
  Python distutils                         yes
  Python yaml                              yes
  Install suricatactl:                     yes
  Install suricatasc:                      yes
  Install suricata-update:                 yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no

  Plugin support (experimental):           yes

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /opt/suritest
  Configuration directory:                 /opt/suritest/etc/suricata/
  Log directory:                           /opt/suritest/var/log/suricata/

  --prefix                                 /opt/suritest
  --sysconfdir                             /opt/suritest/etc
  --localstatedir                          /opt/suritest/var
  --datarootdir                            /opt/suritest/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / g++ (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -ggdb3 -Werror -Wchar-subscripts -fno-strict-aliasing -fstack-protector-all -fsanitize=address -fno-omit-frame-pointer -Wno-unused-parameter -Wno-unused-function -std=c11 -march=native -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS