Project

General

Profile

Support #3079

Suricata Getting Updates

Added by Ralston Champagnie 3 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Target version:
-
Affected Versions:
Effort:
Difficulty:
Label:

Description

I had set up Suricata to get update feed around midnight on my pfSense box which is a 2011 Mac Mini server with a thunderbolt to Ethernet adapter, a broadcom BGE interface. I kept getting this:

Jul 10 00:31:02 php-cgi [Suricata] The Rules update has finished.
Jul 10 00:31:02 SuricataStartup 85868 Suricata START for WAN...
Jul 10 00:31:02 check_reload_status Syncing firewall
Jul 10 00:31:25 kernel 685.365777 [ 254] generic_find_num_desc called, in tx 1024 rx 1024
Jul 10 00:31:25 kernel 685.365802 [ 262] generic_find_num_queues called, in txq 0 rxq 0
Jul 10 00:31:25 kernel 685.365819 [ 760] generic_netmap_dtor Restored native NA 0
Jul 10 00:31:25 kernel 685.367857 [ 254] generic_find_num_desc called, in tx 1024 rx 1024
Jul 10 00:31:25 kernel 685.367878 [ 262] generic_find_num_queues called, in txq 0 rxq 0
Jul 10 00:31:25 kernel 685.367894 [ 760] generic_netmap_dtor Restored native NA 0
Jul 10 00:31:25 kernel 685.367917 [ 254] generic_find_num_desc called, in tx 1024 rx 1024
Jul 10 00:31:25 kernel 685.367933 [ 262] generic_find_num_queues called, in txq 0 rxq 0
Jul 10 06:00:00 php-cgi [Suricata] Checking for updated MaxMind GeoLite2 IP database file...
Jul 10 06:00:00 php-cgi [Suricata] GeoLite2-Country IP database is up-to-date.
Jul 10 06:00:00 php-cgi [Suricata] GeoLite2-Country database update check finished.

A pfSense member led me to this:https://redmine.openinfosecfoundation.org/issues/1688
That link shows a closed case however, it continues. On the link, one person suggested this:
sysctl dev.netmap.admode = 1

I am using Suricata 4.1.4_4

History

#1

Updated by Andreas Herz 3 months ago

  • Tracker changed from Bug to Support
  • Assignee set to Community Ticket
  • Target version set to Soon

Did you try the netmap mode setting?

#2

Updated by Ralston Champagnie 3 months ago

Andreas Herz wrote:

Did you try the netmap mode setting?

When you say try the Netmap mode setting, do you mean this: sysctl dev.netmap.admode = 1?

#3

Updated by Ralston Champagnie 3 months ago

Shell Output - cat /var/log/system.log | grep netmap
Jul 7 13:24:50 NollipfSense kernel: netmap: loaded module
Jul 9 00:30:55 NollipfSense kernel: 255.614367 [ 760] generic_netmap_dtor Restored native NA 0
Jul 9 00:30:55 NollipfSense kernel: 255.616438 [ 760] generic_netmap_dtor Restored native NA 0
Jul 10 00:31:00 NollipfSense kernel: 660.148513 [ 760] generic_netmap_dtor Restored native NA 0
Jul 10 00:31:25 NollipfSense kernel: 685.365819 [ 760] generic_netmap_dtor Restored native NA 0
Jul 10 00:31:25 NollipfSense kernel: 685.367894 [ 760] generic_netmap_dtor Restored native NA 0
Jul 11 00:30:12 NollipfSense kernel: 012.950971 [ 760] generic_netmap_dtor Restored native NA 0
Jul 11 00:30:38 NollipfSense kernel: 038.259726 [ 760] generic_netmap_dtor Restored native NA 0
Jul 11 00:30:38 NollipfSense kernel: 038.261782 [ 760] generic_netmap_dtor Restored native NA 0
Jul 12 00:30:10 NollipfSense kernel: 410.784723 [ 760] generic_netmap_dtor Restored native NA 0
Jul 12 00:30:36 NollipfSense kernel: 436.134532 [ 760] generic_netmap_dtor Restored native NA 0
Jul 12 00:30:36 NollipfSense kernel: 436.136610 [ 760] generic_netmap_dtor Restored native NA 0

%YAML 1.1
---

max-pending-packets: 1024

  1. Runmode the engine should use.
    runmode: autofp
  1. If set to auto, the variable is internally switched to 'router' in IPS
  2. mode and 'sniffer-only' in IDS mode.
    host-mode: auto
  1. Specifies the kind of flow load balancer used by the flow pinned autofp mode.
    autofp-scheduler: active-packets
  1. Daemon working directory
    daemon-directory: /usr/local/etc/suricata/suricata_23163_bge0

default-packet-size: 1514

  1. The default logging directory.
    default-log-dir: /var/log/suricata/suricata_bge023163
  1. global stats configuration
    stats:
    enabled: no
    interval: 10
    #decoder-events: true
    decoder-events-prefix: "decoder.event"
    #stream-events: false
  1. Configure the type of alert (and other) logging.
    outputs:
    1. alert-pf blocking plugin
      - alert-pf:
      enabled: no
      kill-state: yes
      block-drops-only: no
      pass-list: /usr/local/etc/suricata/suricata_23163_bge0/passlist
      block-ip: BOTH
      pf-table: snort2c
    1. a line based alerts log similar to Snort's fast.log
      - fast:
      enabled: yes
      filename: alerts.log
      append: yes
      filetype: regular
    1. alert output for use with Barnyard2
      - unified2-alert:
      enabled: no
      filename: unified2.alert
      limit: 32mb
      sensor-id: 0
      xff:
      enabled: no
    - http-log:
    enabled: yes
    filename: http.log
    append: yes
    extended: yes
    filetype: regular
    - pcap-log:
    enabled: no
    filename: log.pcap
    limit: 32mb
    max-files: 1000
    mode: normal
    - tls-log:
    enabled: no
    filename: tls.log
    extended: yes
    - tls-store:
    enabled: no
    certs-log-dir: certs
    - stats:
    enabled: yes
    filename: stats.log
    append: no
    totals: yes
    threads: no
    #null-values: yes
    - syslog:
    enabled: no
    identity: suricata
    facility: local1
    level: notice
    - drop:
    enabled: no
    filename: drop.log
    append: yes
    filetype: regular
    - file-store:
    version: 2
    enabled: no
    log-dir: files
    force-magic: no
    #force-hash: [md5]
    #waldo: file.waldo
    - file-log:
    enabled: no
    filename: files-json.log
    append: yes
    filetype: regular
    force-magic: no
    #force-hash: [md5]
    - eve-log:
    enabled: no
    filetype: regular
    filename: eve.json
    redis:
    server: 127.0.0.1
    port: 6379
    mode: list
    key: "suricata"
    identity: "suricata"
    facility: local1
    level: notice
    xff:
    enabled: no
    mode: extra-data
    deployment: reverse
    header: X-Forwarded-For
    types:
    - alert:
    payload: yes # enable dumping payload in Base64
    payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
    payload-printable: yes # enable dumping payload in printable (lossy) format
    packet: yes # enable dumping of packet (without stream segments)
    http-body: yes # enable dumping of http body in Base64
    http-body-printable: yes # enable dumping of http body in printable format
    tagged-packets: yes # enable logging of tagged packets for rules using the 'tag' keyword
    - http:
    extended: yes
    custom: [accept, accept-charset, accept-datetime, accept-encoding, accept-language, accept-range, age, allow, authorization, cache-control, connection, content-encoding, content-language, content-length, content-location, content-md5, content-range, content-type, cookie, date, dnt, etags, from, last-modified, link, location, max-forwards, origin, pragma, proxy-authenticate, proxy-authorization, range, referrer, refresh, retry-after, server, set-cookie, te, trailer, transfer-encoding, upgrade, vary, via, warning, www-authenticate, x-authenticated-user, x-flash-version, x-forwarded-proto, x-requested-with]
    - dns:
    version: 2
    query: yes
    answer: yes
    - tls:
    extended: yes
    - dhcp:
    extended: no
    - files:
    force-magic: no
    - ssh
    - nfs
    - smb
    - krb5
    - ikev2
    - tftp
    - smtp:
    extended: yes
    custom: [bcc, received, reply-to, x-mailer, x-originating-ip]
    md5: [subject]
    - drop:
    alerts: yes
    flows: all
  1. Magic file. The extension .mgc is added to the value here.
    magic-file: /usr/share/misc/magic
  1. GeoLite2 IP geo-location database file path and filename.
    geoip-database: /usr/local/share/suricata/GeoLite2/GeoLite2-Country.mmdb
  1. Specify a threshold config file
    threshold-file: /usr/local/etc/suricata/suricata_23163_bge0/threshold.config

detect-engine:
- profile: high
- sgh-mpm-context: auto
- inspection-recursion-limit: 3000
- delayed-detect: no

  1. Suricata is multi-threaded. Here the threading can be influenced.
    threading:
    set-cpu-affinity: no
    detect-thread-ratio: 1.0
  1. Luajit has a strange memory requirement, it's 'states' need to be in the
  2. first 2G of the process' memory. #
  3. 'luajit.states' is used to control how many states are preallocated.
  4. State use: per detect script: 1 per detect thread. Per output script: 1 per
  5. script.
    luajit:
    states: 128
  1. Multi pattern algorithm
  2. The default mpm-algo value of "auto" will use "hs" if Hyperscan is
  3. available, "ac" otherwise.
    mpm-algo: auto
  1. Single pattern algorithm
  2. The default of "auto" will use "hs" if available, otherwise "bm".
    spm-algo: auto
  1. Defrag settings:
    defrag:
    memcap: 33554432
    hash-size: 65536
    trackers: 65535
    max-frags: 65535
    prealloc: yes
    timeout: 60
  1. Flow settings:
    flow:
    memcap: 33554432
    hash-size: 65536
    prealloc: 10000
    emergency-recovery: 30
    prune-flows: 5
  1. This option controls the use of vlan ids in the flow (and defrag)
  2. hashing.
    vlan:
    use-for-tracking: true
  1. Specific timeouts for flows.
    flow-timeouts:
    default:
    new: 30
    established: 300
    closed: 0
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 0
    tcp:
    new: 60
    established: 3600
    closed: 120
    emergency-new: 10
    emergency-established: 300
    emergency-closed: 20
    udp:
    new: 30
    established: 300
    emergency-new: 10
    emergency-established: 100
    icmp:
    new: 30
    established: 300
    emergency-new: 10
    emergency-established: 100

stream:
memcap: 512000000
checksum-validation: no
inline: auto
prealloc-sessions: 32768
midstream: false
async-oneside: false
max-synack-queued: 5
reassembly:
memcap: 67108864
depth: 1048576
toserver-chunk-size: 2560
toclient-chunk-size: 2560

  1. Host table is used by tagging and per host thresholding subsystems.
    host:
    hash-size: 4096
    prealloc: 1000
    memcap: 33554432
  1. Host specific policies for defragmentation and TCP stream reassembly.
    host-os-policy:
    bsd: [0.0.0.0/0]
  1. Logging configuration. This is not about logging IDS alerts, but
  2. IDS output about what its doing, errors, etc.
    logging:
    1. This value is overriden by the SC_LOG_LEVEL env var.
      default-log-level: info
      default-log-format: "%t - <%d> -- "
    1. Define your logging outputs.
      outputs:
      - console:
      enabled: yes
      - file:
      enabled: yes
      filename: /var/log/suricata/suricata_bge023163/suricata.log
      - syslog:
      enabled: no
      facility: off
      format: "[%i] <%d> -- "
  1. IPS Mode Configuration
  2. Netmap
    netmap:
    - interface: default
    threads: auto
    copy-mode: ips
    disable-promisc: no
    checksum-checks: auto
    - interface: bge0
    copy-iface: bge0+
    - interface: bge0+
    copy-iface: bge0

legacy:
uricontent: enabled

default-rule-path: /usr/local/etc/suricata/suricata_23163_bge0/rules
rule-files:
- suricata.rules

classification-file: /usr/local/etc/suricata/suricata_23163_bge0/classification.config
reference-config-file: /usr/local/etc/suricata/suricata_23163_bge0/reference.config

  1. Holds variables that would be used by the engine.
    vars:
    1. Holds the address group vars that would be passed in a Signature.
      address-groups:
      HOME_NET: "[10.10.10.1/32,68.226.180.1/32,68.226.181.34/32,127.0.0.1/32,192.168.1.0/24,208.67.220.220/32,208.67.222.222/32,::1/128,fe80::aa60:b6ff:fe23:1134/128,fe80::ca2a:14ff:fe57:d2dc/128]"
      EXTERNAL_NET: "[!10.10.10.1/32,!68.226.180.1/32,!68.226.181.34/32,!127.0.0.1/32,!192.168.1.0/24,!208.67.220.220/32,!208.67.222.222/32,::1/128,fe80::aa60:b6ff:fe23:1134/128,fe80::ca2a:14ff:fe57:d2dc/128]"
      DNS_SERVERS: "$HOME_NET"
      SMTP_SERVERS: "$HOME_NET"
      HTTP_SERVERS: "$HOME_NET"
      SQL_SERVERS: "$HOME_NET"
      TELNET_SERVERS: "$HOME_NET"
      DNP3_SERVER: "$HOME_NET"
      DNP3_CLIENT: "$HOME_NET"
      MODBUS_SERVER: "$HOME_NET"
      MODBUS_CLIENT: "$HOME_NET"
      ENIP_SERVER: "$HOME_NET"
      ENIP_CLIENT: "$HOME_NET"
      FTP_SERVERS: "$HOME_NET"
      SSH_SERVERS: "$HOME_NET"
      AIM_SERVERS: "64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/24,205.188.248.0/24"
      SIP_SERVERS: "$HOME_NET"
    1. Holds the port group vars that would be passed in a Signature.
      port-groups:
      FTP_PORTS: "21"
      HTTP_PORTS: "80"
      ORACLE_PORTS: "1521"
      SSH_PORTS: "22"
      SHELLCODE_PORTS: "!80"
      DNP3_PORTS: "20000"
      FILE_DATA_PORTS: "$HTTP_PORTS,110,143"
      SIP_PORTS: "5060,5061,5600"
  1. Set the order of alerts based on actions
    action-order:
    - pass
    - drop
    - reject
    - alert
  1. IP Reputation
  1. Limit for the maximum number of asn1 frames to decode (default 256)
    asn1-max-frames: 256

engine-analysis:
rules-fast-pattern: yes
rules: yes

#recursion and match limits for PCRE where supported
pcre:
match-limit: 3500
match-limit-recursion: 1500

  1. Holds details on the app-layer. The protocols section details each protocol.
    app-layer:
    protocols:
    dcerpc:
    enabled: yes
    dhcp:
    enabled: yes
    dnp3:
    enabled: yes
    detection-ports:
    dp: 20000
    dns:
    global-memcap: 16777216
    state-memcap: 524288
    request-flood: 500
    tcp:
    enabled: yes
    detection-ports:
    dp: 53
    udp:
    enabled: yes
    detection-ports:
    dp: 53
    ftp:
    enabled: yes
    http:
    enabled: yes
    memcap: 67108864
    ikev2:
    enabled: yes
    imap:
    enabled: detection-only
    krb5:
    enabled: yes
    modbus:
    enabled: yes
    request-flood: 500
    detection-ports:
    dp: 502
    stream-depth: 0
    msn:
    enabled: detection-only
    nfs:
    enabled: yes
    ntp:
    enabled: yes
    tls:
    enabled: yes
    detection-ports:
    dp: 443
    ja3-fingerprints: off
    encrypt-handling: default
    smb:
    enabled: yes
    detection-ports:
    dp: 139, 445
    smtp:
    enabled: yes
    mime:
    decode-mime: no
    decode-base64: yes
    decode-quoted-printable: yes
    header-value-depth: 2000
    extract-urls: yes
    body-md5: no
    inspected-tracker:
    content-limit: 100000
    content-inspect-min-size: 32768
    content-inspect-window: 4096
    ssh:
    enabled: yes
    tftp:
    enabled: yes
###########################################################################
  1. Configure libhtp.
    libhtp:
    default-config:
    personality: IDS
    request-body-limit: 4096
    response-body-limit: 4096
    double-decode-path: no
    double-decode-query: no
    uri-include-all: no

coredump:
max-dump: unlimited

  1. Suricata user pass through configuration
#4

Updated by Ralston Champagnie 3 months ago

Andreas Herz wrote:

Did you try the netmap mode setting?

Please close this report/case as it's not a bug...thank you!

#5

Updated by Andreas Herz 3 months ago

  • Status changed from New to Closed

Would still be helpful if you add the explanation as well :)

#6

Updated by Victor Julien 3 months ago

  • Target version deleted (Soon)

Also available in: Atom PDF