Suricata

Suricata is segfaulting during a memcpy in app-layer-dcerpc.c StubDataParser when long running smb flows contain dcerpc traffic. I think the stub_data_buffer_len variable is the culprit.

I modified the app-layer-smb.c DataParser, app-layer-dcerpc.c DCERPCParser and StubDataParser methods to pass along the flow_hash and it looks something like this:
(printing in StubDataParser)

13/7/2017 -- 20:35:02 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 0, dcerpc->fragged: 0, dcerpc->transaction_id: 1
13/7/2017 -- 20:35:02 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 1000, dcerpc->fragged: 1, dcerpc->transaction_id: 2
13/7/2017 -- 20:35:02 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 2024, dcerpc->fragged: 1, dcerpc->transaction_id: 3
13/7/2017 -- 20:35:02 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 3048, dcerpc->fragged: 1, dcerpc->transaction_id: 4
...
13/7/2017 -- 20:36:09 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 15302088, dcerpc->fragged: 1, dcerpc->transaction_id: 18626
13/7/2017 -- 20:36:09 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 15302272, dcerpc->fragged: 0, dcerpc->transaction_id: 18627
13/7/2017 -- 20:36:09 - <Error> - [ERRCODE: SC_ERR_DCERPC(172)] - flow_hash: 459222427, stub_data_buffer_len: 15303272, dcerpc->fragged: 1, dcerpc->transaction_id: 18628

Printing the stub_data_buffer_len with gdb from the coredump shows that it eventually reaches 4294967256. This value was consistent across multiple core dumps.

$ gdb /usr/local/bin/suricata core.10416
...
bt full 2
#0  0x00007febd6540f8e in __memcpy_ssse3_back () from /lib64/libc.so.6
No symbol table info available.
#1  0x000000000041837a in StubDataParser (dcerpc=dcerpc@entry=0x7feb3fddf7b8, input=input@entry=0x7feb04a9f5d0 "", input_len=input_len@entry=1024) at app-layer-dcerpc.c:1236
        stub_data_buffer = 0x7feb3fddf830
        stub_data_buffer_len = 0x7feb3fddf838
        stub_data_fresh = 0x7feb3fddf83c "" 
        stub_len = 1024
        ptmp = <optimized out>
        __FUNCTION__ = "StubDataParser" 
(More stack frames follow...)

x/u 0x7feb3fddf838
0x7feb3fddf838: 4294967256

I have tried taking a pcap of a 5 minute interval around when the segfault occurs, but reading or replaying it to Suricata does not recreate the error. I think this only occurs after a long time of building up state, about once every 24 hours in my case.

Setting smb to "detection-only" in the yaml stops the segfault from occurring.
While trying to debug this I also noticed that DataParser in app-layer-smb.c checks the return value of DCERPCParser against -1, although it seems that DCERPCParser only returns 0 on failure. But changing this did not stop the segfaults.

I can't share a pcap or coredump sadly, sorry, but I attached gdb.txt with some thread info.

Running Suricata with pf_ring with this build-info:

$ suricata --build-info
This is Suricata version 3.2 RELEASE
Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LIBJANSSON TLS
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 4.8.5 20150623 (Red Hat 4.8.5-11), C version 199901
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.23, linked against LibHTP v0.5.23

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         yes
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             no
  libluajit:                               no
  libgeoip:                                no
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  CUDA enabled:                            no
  Hyperscan support:                       no
  Libnet support:                          no

  Suricatasc install:                      yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr/local
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr/local
  --sysconfdir                             /etc
  --localstatedir                          /var

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native
  PCAP_CFLAGS
  SECCFLAGS