Project

General

Profile

Actions

Bug #1817

closed

Suricata 3.1RC1 crashing

Added by Marius Flage over 8 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
High
Assignee:
-
Target version:
-
Affected Versions:
Effort:
Difficulty:
Label:

Description

Hi!

Just recently upgraded to 3.1RC1 and it segfaults on me.

This is the backtrace:

#0  0x00007f42ad9f29f5 in TCPCalculateChecksum (tlen=680, pkt=<optimized out>, shdr=<optimized out>) at decode-tcp.h:198
#1  StreamTcpValidateChecksum (p=<optimized out>) at stream-tcp.c:4672
#2  StreamTcp (tv=tv@entry=0x7f42aff9c280, p=p@entry=0x7f4290193ca0, data=0x7f42400013e0, pq=pq@entry=0x7f42400008d8, postpq=0x7f429c3202b4, postpq@entry=0x0) at stream-tcp.c:4868
#3  0x00007f42ad988965 in FlowWorker (tv=0x7f42aff9c280, p=0x7f4290193ca0, data=0x7f42400008c0, preq=0x7f42aff9c3d0, unused=<optimized out>) at flow-worker.c:182
#4  0x00007f42ada01704 in TmThreadsSlotVarRun (tv=tv@entry=0x7f42aff9c280, p=p@entry=0x7f4290193ca0, slot=slot@entry=0x7f42aff9c390) at tm-threads.c:132
#5  0x00007f42ada04586 in TmThreadsSlotVar (td=0x7f42aff9c280) at tm-threads.c:593
#6  0x00007f42ac137184 in start_thread (arg=0x7f424affd700) at pthread_create.c:312
#7  0x00007f42ab27037d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Output of build-info:

# /usr/bin/suricata --build-info
This is Suricata version 3.1RC1 RELEASE
Features: NFQ PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS 
SIMD support: none
Atomic intrisics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 4.8.4, C version 199901
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.20, linked against LibHTP v0.5.20

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         no
  NFQueue support:                         yes
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  libgeoip:                                yes
  Non-bundled htp:                         yes
  Old barnyard2 support:                   no
  CUDA enabled:                            no
  Hyperscan support:                       no
  Libnet support:                          yes

  Suricatasc install:                      yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     no
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     yes
  GCC march native enabled:                no
  GCC Profile enabled:                     no
  Position Independent Executable enabled: yes
  CFLAGS                                   -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                                -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security

This is the version that's installed as a package on Ubuntu 14.04, from the following APT source:

deb http://ppa.launchpad.net/oisf/suricata-beta/ubuntu trusty main

I can provide the core dump if needed, but it's rather big.


Related issues 1 (0 open1 closed)

Related to Suricata - Bug #1848: crash if disk is fullClosedActions
Actions #1

Updated by Marius Flage over 8 years ago

The command to run Suricata is:

/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v

The same happens if I run Suricata in non-daemon mode:

21/6/2016 -- 12:08:44 - <Info> - All AFP capture threads are running.
Segmentation fault (core dumped)
Actions #2

Updated by Marius Flage over 8 years ago

Hoy! Update.

Figured out what caused the segfault. The directory used for pcap dump was full and this caused the segfault. Strangely enough, this didn't happen if I build Suricata from source.

I currently have the following outputs defined:

outputs:


  # a line based alerts log similar to Snort's fast.log
  - fast:
      enabled: yes
      filename: fast.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata" 
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert:
            # payload: yes           # enable dumping payload in Base64
            # payload-printable: yes # enable dumping payload in printable (lossy) format
            # packet: yes            # enable dumping of packet (without stream segments)
            # http: yes              # enable dumping of http fields
            # tls: yes               # enable dumping of tls fields
            # ssh: yes               # enable dumping of ssh fields

            # HTTP X-Forwarded-For support by adding an extra field or overwriting
            # the source or destination IP address (depending on flow direction)
            # with the one reported in the X-Forwarded-For HTTP header. This is
            # helpful when reviewing alerts for traffic that is being reverse
            # or forward proxied.
            xff:
              enabled: no
              # Two operation modes are available, "extra-data" and "overwrite".
              mode: extra-data
              # Two proxy deployments are supported, "reverse" and "forward". In
              # a "reverse" deployment the IP address used is the last one, in a
              # "forward" deployment the first IP address is used.
              deployment: reverse
              # Header name where the actual IP address will be reported, if more
              # than one IP address is present, the last IP address will be the
              # one taken into consideration.
              header: X-Forwarded-For
#        - http:
#            extended: yes     # enable this for extended logging information
            # custom allows additional http fields to be included in eve-log
            # the example below adds three additional fields when uncommented
            #custom: [Accept-Encoding, Accept-Language, Authorization]
#        - dns

#        - tls:
#            extended: yes     # enable this for extended logging information
#        - files:
#            force-magic: no   # force logging magic on all logged files
#            force-md5: no     # force logging of md5 checksums
        #- drop:
        #    alerts: no       # log alerts that caused drops
#        - smtp
#        - ssh
        # bi-directional flows
        #- flow
        # uni-directional flows
        #- newflow
  # alert output for use with Barnyard2
  - unified2-alert:
      enabled: yes
      filename: unified2.alert

      # File size limit.  Can be specified in kb, mb, gb.  Just a number
      # is parsed as bytes.
      #limit: 32mb

      # Sensor ID field of unified2 alerts.
      #sensor-id: 0

      # HTTP X-Forwarded-For support by adding the unified2 extra header or
      # overwriting the source or destination IP address (depending on flow
      # direction) with the one reported in the X-Forwarded-For HTTP header.
      # This is helpful when reviewing alerts for traffic that is being reverse
      # or forward proxied.
      xff:
        enabled: no
        # Two operation modes are available, "extra-data" and "overwrite". Note
        # that in the "overwrite" mode, if the reported IP address in the HTTP
        # X-Forwarded-For header is of a different version of the packet
        # received, it will fall-back to "extra-data" mode.
        mode: extra-data
        # Two proxy deployments are supported, "reverse" and "forward". In
        # a "reverse" deployment the IP address used is the last one, in a
        # "forward" deployment the first IP address is used.
        deployment: reverse
        # Header name where the actual IP address will be reported, if more
        # than one IP address is present, the last IP address will be the
        # one taken into consideration.
        header: X-Forwarded-For

  # a line based log of HTTP requests (no alerts)
  - http-log:
      enabled: no
      filename: http.log
      append: yes
     #extended: yes     # enable this for extended logging information
      #custom: yes       # enabled the custom logging format (defined by customformat)
      #customformat: "%{%D-%H:%M:%S}t.%z %{X-Forwarded-For}i %H %m %h %u %s %B %a:%p -> %A:%P" 
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # a line based log of TLS handshake parameters (no alerts)
  - tls-log:
      enabled: no  # Log TLS connections.
      filename: tls.log # File to store TLS logs.
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
      #extended: yes # Log extended information like fingerprint
      certs-log-dir: certs # directory to store the certificates files
  # a line based log of DNS requests and/or replies (no alerts)
  - dns-log:
      enabled: no
      filename: dns.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # Packet log... log packets in pcap format. 3 modes of operation: "normal" 
  # "multi" and "sguil".
  #
  # In normal mode a pcap file "filename" is created in the default-log-dir,
  # or are as specified by "dir".
  # In multi mode, a file is created per thread. This will perform much
  # better, but will create multiple files where 'normal' would create one.
  # In multi mode the filename takes a few special variables:
  # - %n -- thread number
  # - %i -- thread id
  # - %t -- timestamp (secs or secs.usecs based on 'ts-format'
  # E.g. filename: pcap.%n.%t
  #
  # Note that it's possible to use directories, but the directories are not
  # created by Suricata. E.g. filename: pcaps/%n/log.%s will log into the
  # per thread directory.
  #
  # Also note that the limit and max-files settings are enforced per thread.
  # So the size limit when using 8 threads with 1000mb files and 2000 files
  # is: 8*1000*2000 ~ 16TiB.
  #
  # In Sguil mode "dir" indicates the base directory. In this base dir the
  # pcaps are created in th directory structure Sguil expects:
  #
  # $sguil-base-dir/YYYY-MM-DD/$filename.<timestamp>
  #
  # By default all packets are logged except:
  # - TCP streams beyond stream.reassembly.depth
  # - encrypted streams after the key exchange
  #
  - pcap-log:
      enabled:  yes
      filename: log.pcap

      # File size limit.  Can be specified in kb, mb, gb.  Just a number
      # is parsed as bytes.
      limit: 1000mb

      # If set to a value will enable ring buffer mode. Will keep Maximum of "max-files" of size "limit" 
      max-files: 450

      mode: normal # normal, multi or sguil.
      sguil-base-dir: /data/pcaps
      #ts-format: usec # sec or usec second format (default) is filename.sec usec is filename.sec.usec
      use-stream-depth: no #If set to "yes" packets seen after reaching stream inspection depth are ignored. "no" logs all packets
      honor-pass-rules: no # If set to "yes", flows in which a pass rule matched will stopped being logged.

  # a full alerts log containing much information for signature writers
  # or for investigating suspected false positives.
  - alert-debug:
      enabled: no
      filename: alert-debug.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # alert output to prelude (http://www.prelude-technologies.com/) only
  # available if Suricata has been compiled with --enable-prelude
  - alert-prelude:
      enabled: no
      profile: suricata
      log-packet-content: no
      log-packet-header: yes

  # Stats.log contains data from various counters of the suricata engine.
  - stats:
      enabled: yes
      filename: stats.log

  # a line based alerts log similar to fast.log into syslog
  - syslog:
      enabled: no
      # reported identity to syslog. If ommited the program name (usually
      # suricata) will be used.
      #identity: "suricata" 
      facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug

  # a line based information for dropped packets in IPS mode
  - drop:
      enabled: no
      filename: drop.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

  # output module to store extracted files to disk
  #
  # The files are stored to the log-dir in a format "file.<id>" where <id> is
  # an incrementing number starting at 1. For each file "file.<id>" a meta
  # file "file.<id>.meta" is created.
  #
  # File extraction depends on a lot of things to be fully done:
  # - stream reassembly depth. For optimal results, set this to 0 (unlimited)
  # - http request / response body sizes. Again set to 0 for optimal results.
  # - rules that contain the "filestore" keyword.
  - file-store:
     enabled: no       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: no   # force logging magic on all stored files
      force-md5: no     # force logging of md5 checksums
      #waldo: file.waldo # waldo file to store the file_id across runs

  # output module to log files tracked in a easily parsable json format
  - file-log:
      enabled: no
      filename: files-json.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

      force-magic: no   # force logging magic on all logged files
      force-md5: no     # force logging of md5 checksums

  # Log TCP data after stream normalization
  # 2 types: file or dir. File logs into a single logfile. Dir creates
  # 2 files per TCP session and stores the raw TCP data into them.
  # Using 'both' will enable both file and dir modes.
  #
  # Note: limited by stream.depth
  - tcp-data:
      enabled: no
      type: file
      filename: tcp-data.log
  # Log HTTP body data after normalization, dechunking and unzipping.
  # 2 types: file or dir. File logs into a single logfile. Dir creates
  # 2 files per HTTP session and stores the normalized data into them.
  # Using 'both' will enable both file and dir modes.
  #
  # Note: limited by the body limit settings
  - http-body-data:
      enabled: no
      type: file
      filename: http-data.log

  # Lua Output Support - execute lua script to generate alert and event
  # output.
  # Documented at:
  # https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Lua_Output
  - lua:
      enabled: no
      #scripts-dir: /etc/suricata/lua-output/
      scripts:
      #   - script1.lua

Hopefully this will be helpful to someone :)

Actions #3

Updated by Andreas Herz over 8 years ago

Should we try to intercept that?

Actions #4

Updated by Andreas Herz over 8 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD
Actions #5

Updated by Victor Julien over 6 years ago

  • Related to Bug #1848: crash if disk is full added
Actions #6

Updated by Victor Julien almost 6 years ago

  • Status changed from New to Closed
  • Assignee deleted (OISF Dev)
  • Target version deleted (TBD)

Closing in favor of #1848

Actions

Also available in: Atom PDF