Project

General

Profile

Actions

Feature #2227

open

more detailed output about number of threads created

Added by Peter Manev almost 4 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Target version:
Effort:
low
Difficulty:
low
Label:

Description

On a 8 core machine -

sudo suricata -S /dev/null  -vvv --pcap=eth0 --set pcap.0.threads=auto --runmode=autofp --set threading.detect-thread-ratio=1
...
26754] 11/10/2017 -- 11:29:14 - (util-conf.c:109) <Info> (ConfUnixSocketIsEnable) -- Running in live mode, activating unix socket
[26754] 11/10/2017 -- 11:29:14 - (unix-manager.c:124) <Info> (UnixNew) -- Using unix socket file '/var/run/suricata/suricata-command.socket'
[26754] 11/10/2017 -- 11:29:14 - (tm-threads.c:2178) <Notice> (TmThreadWaitOnThreadInit) -- all 9 packet processing threads, 4 management threads initialized, engine started.
sudo suricata -S /dev/null  -vvv --pcap=eth0 --set pcap.0.threads=auto --runmode=autofp --set threading.detect-thread-ratio=2
....
[9520] 11/10/2017 -- 20:45:46 - (runmode-pcap.c:292) <Info> (RunModeIdsPcapAutoFp) -- RunModeIdsPcapAutoFp initialised
[9520] 11/10/2017 -- 20:45:46 - (flow-manager.c:828) <Config> (FlowManagerThreadSpawn) -- using 1 flow manager threads
[9520] 11/10/2017 -- 20:45:46 - (flow-manager.c:992) <Config> (FlowRecyclerThreadSpawn) -- using 1 flow recycler threads
[9520] 11/10/2017 -- 20:45:46 - (util-conf.c:109) <Info> (ConfUnixSocketIsEnable) -- Running in live mode, activating unix socket
[9520] 11/10/2017 -- 20:45:46 - (unix-manager.c:124) <Info> (UnixNew) -- Using unix socket file '/var/run/suricata/suricata-command.socket'
[9520] 11/10/2017 -- 20:45:46 - (tm-threads.c:2178) <Notice> (TmThreadWaitOnThreadInit) -- all 17 packet processing threads, 4 management threads initialized, engine started.

I would expect the first run to have 8 packet processing threads and the second 16 packet processing threads.

Suricata info -

This is Suricata version 4.0.0-dev (rev 11806875)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS MAGIC 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 6.3.0 20170516, C version 199901
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.25, linked against LibHTP v0.5.25

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  libgeoip:                                yes
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  CUDA enabled:                            no
  Hyperscan support:                       no
  Libnet support:                          yes

  Rust support (experimental):             no
  Experimental Rust parsers:               no
  Rust strict mode:                        no

  Suricatasc install:                      yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     yes
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                                

Actions #1

Updated by Peter Manev almost 4 years ago

  • Subject changed from detect-thread-ratio has produces unexpected results with autofp to detect-thread-ratio produces unexpected results with autofp
Actions #2

Updated by Andreas Herz almost 4 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD
Actions #3

Updated by Victor Julien almost 4 years ago

Seems to working as intended. There is 1 pcap receive thread, and then 8 and 16 'workers' as configured by the ratio. The ratio is applies as core/ht count * ratio. So this should be a box with 8 cores/hyperthreads.

Actions #4

Updated by Peter Manev almost 4 years ago

Yes indeed - I forgot that 1 receive thread!
Wonder if it is worthed adding that to the output with autofp? (aka something like "all 1 pcap receive thread, 16 packet processing threads, 4 management threads initialized,")

Actions #5

Updated by Victor Julien almost 4 years ago

  • Tracker changed from Bug to Feature
  • Subject changed from detect-thread-ratio produces unexpected results with autofp to more detailed output about number of threads created
  • Assignee changed from OISF Dev to Anonymous
Actions #6

Updated by Victor Julien about 3 years ago

  • Effort set to low
  • Difficulty set to low
Actions #7

Updated by Andreas Herz over 2 years ago

  • Assignee set to Community Ticket
Actions

Also available in: Atom PDF