Project

General

Profile

Actions

Feature #4226

open

bsize: apply as depth to patterns

Added by Brandon Murphy almost 4 years ago. Updated 7 months ago.

Status:
Assigned
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:
Needs backport

Description

When reviewing rule profiling output of comparing the speed of bsize:x; checks to using depth:x; isdataat:!1,relative; in effort to achieve an "exact match" of a buffer, it has been observed that bsize variants of rules often consume considerably more ticks than alternative methods.

Tested on 7.0.0-dev (372fc2673 2020-12-11) with the default suricata.yaml, with minor adjustments for non-json output of the rule profiling output.

command used for testing

suricata -c suricata.yaml -S test.rules -r a564fbcf-e41d-494c-9ad8-7e44e22a03d9.pcap -l /tmp/perf_test/perf_1/

pcap from https://app.any.run/tasks/a564fbcf-e41d-494c-9ad8-7e44e22a03d9/#

test signatures

alert dns any any -> any any (msg:"bsize test"; dns.query; content:"yundol0727.kro.kr"; bsize:17; sid:1;)
alert dns any any -> any any (msg:"depth and isdataat test"; dns.query; content:"yundol0727.kro.kr"; depth:17; isdataat:!1,relative; sid:2;)
alert dns any any -> any any (msg:"depth and endswith test"; dns.query; content:"yundol0727.kro.kr"; depth:17; endswith; sid:3;)

It's been observed on other buffers as well, just using dns.query as a quick example.

I ran the above signatures through on the same pcap 3 times and have recorded the output of the

First Run

  --------------------------------------------------------------------------
  Date: 12/18/2020 -- 04:12:47. Sorted by: ticks.
  --------------------------------------------------------------------------
   Num      Rule         Gid      Rev      Ticks        %      Checks   Matches  Max Ticks   Avg Ticks   Avg Match   Avg No Match
  -------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- -------------- 
  1        1            1        0        45045        66.10  1        1        45045       45045.00    45045.00    0.00       
  2        2            1        0        12123        17.79  1        1        12123       12123.00    12123.00    0.00       
  3        3            1        0        10983        16.12  1        1        10983       10983.00    10983.00    0.00       

Second Run

  --------------------------------------------------------------------------
  Date: 12/18/2020 -- 04:17:23. Sorted by: ticks.
  --------------------------------------------------------------------------
   Num      Rule         Gid      Rev      Ticks        %      Checks   Matches  Max Ticks   Avg Ticks   Avg Match   Avg No Match
  -------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- -------------- 
  1        1            1        0        26637        53.87  1        1        26637       26637.00    26637.00    0.00       
  2        2            1        0        11772        23.81  1        1        11772       11772.00    11772.00    0.00       
  3        3            1        0        11040        22.33  1        1        11040       11040.00    11040.00    0.00       

Third Run

  --------------------------------------------------------------------------
  Date: 12/18/2020 -- 04:18:08. Sorted by: ticks.
  --------------------------------------------------------------------------
   Num      Rule         Gid      Rev      Ticks        %      Checks   Matches  Max Ticks   Avg Ticks   Avg Match   Avg No Match
  -------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- -------------- 
  1        1            1        0        27219        53.34  1        1        27219       27219.00    27219.00    0.00       
  2        2            1        0        12696        24.88  1        1        12696       12696.00    12696.00    0.00       
  3        3            1        0        11115        21.78  1        1        11115       11115.00    11115.00    0.00       

Build Info

uricata --build-info
This is Suricata version 7.0.0-dev (372fc2673 2020-12-11)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON PROFILING TLS TLS_C11 MAGIC RUST 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 7.5.0, C version 201112
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.36, linked against LibHTP v0.5.36

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  GeoIP2 support:                          no
  Non-bundled htp:                         no
  Hyperscan support:                       no
  Libnet support:                          yes
  liblz4 support:                          yes

  Rust support:                            yes
  Rust strict mode:                        no
  Rust compiler path:                      /usr/bin/rustc
  Rust compiler version:                   rustc 1.43.0
  Cargo path:                              /usr/bin/cargo
  Cargo version:                           cargo 1.43.0
  Cargo vendor:                            yes

  Python support:                          no
  Python path:                             not set
  Python distutils                         no
  Python yaml                              no
  Install suricatactl:                     requires python
  Install suricatasc:                      requires python
  Install suricata-update:                 not bundled

  Profiling enabled:                       yes
  Profiling locks enabled:                 no

  Plugin support (experimental):           yes

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr/local
  Configuration directory:                 /usr/local/etc/suricata/
  Log directory:                           /usr/local/var/log/suricata/

  --prefix                                 /usr/local
  --sysconfdir                             /usr/local/etc
  --localstatedir                          /usr/local/var
  --datarootdir                            /usr/local/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / g++ (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -std=c11 -march=native -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                                


Files


Related issues 2 (2 open0 closed)

Related to Suricata - Optimization #6375: detect: merge urilen and bsize implementationsNewOISF DevActions
Related to Suricata - Bug #6894: bsize validation FP on content negation with hex encoded 0d 0aNewOISF DevActions
Actions

Also available in: Atom PDF