Suricata

When starting suricata with a very large BPF, it fails and starts a loop which causes Suricata to eventually use all memory.

The configuration I used is:

af-packet:
  - interface: "eth0" 
    cluster-id: 2
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    threads: 1
    ring-size: 609144
    bpf-filter: "not (host 192.168.0.1 and portrange 8000-8001)  
    and not (host 192.168.20.3 and portrange 8000-8001)  
    and not (host 192.168.22.15 and port (4001 or 4002 or 4008))  
    and not (host 192.168.22.15 and port (4001 or 4002 or 4008))  
    and not (host 192.168.22.1 and port 443)  
    and not (host 192.168.20.3 and port 902)  
    and not (host 192.168.20.7 and port 902)  
    and not (host 192.168.20.8 and port 902)  

    and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" 

The output from Suricata is as follows:

30/6/2017 -- 08:34:51 - <Info> - Including configuration file /srv/suricata/bond0/suricata.yaml.
30/6/2017 -- 08:34:51 - <Notice> - This is Suricata version 3.2.2 RELEASE
30/6/2017 -- 08:34:51 - <Info> - CPUs/cores online: 1
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DNS_CONFIG(240)] - no DNS UDP config found, enabling DNS detection on port 53.
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DNS_CONFIG(240)] - no DNS TCP config found, enabling DNS detection on port 53.
30/6/2017 -- 08:34:51 - <Info> - No 'host-mode': suricata is in IDS mode, using default setting 'sniffer-only'
30/6/2017 -- 08:34:51 - <Info> - 1 rule files processed. 4 rules successfully loaded, 0 rules failed
30/6/2017 -- 08:34:51 - <Info> - 4 signatures processed. 0 are IP-only rules, 4 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
30/6/2017 -- 08:34:51 - <Info> - Threshold config parsed: 0 rule(s) found
30/6/2017 -- 08:34:51 - <Info> - dropped the caps for main thread
30/6/2017 -- 08:34:51 - <Info> - unified2-alert "limit" value of 1 assumed to be pre-1.2 style: setting limit to 1mb
30/6/2017 -- 08:34:51 - <Info> - Unified2-alert initialized: filename unified2.alert, limit 1 MB
30/6/2017 -- 08:34:51 - <Info> - eve-log output device (unix_dgram) initialized: /var/run/suricata/bond0_output.sock
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DEPRECATED_CONF(274)] - deprecated 'force-md5' option found. Please use 'force-hash: [md5]' instead
30/6/2017 -- 08:34:51 - <Info> - forcing md5 calculation for logged files
30/6/2017 -- 08:34:51 - <Info> - Going to use 1 thread(s)
30/6/2017 -- 08:34:51 - <Info> - Using unix socket file '/var/run/suricata//bond0'
30/6/2017 -- 08:34:51 - <Notice> - all 1 packet processing threads, 4 management threads initialized, engine started.
30/6/2017 -- 08:34:52 - <Info> - Using BPF 'not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)' on iface 'eth0'
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Failed to attach filter: Cannot allocate memory
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Set AF_PACKET bpf filter "not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" failed.
30/6/2017 -- 08:34:52 - <Info> - All AFP capture threads are running.
30/6/2017 -- 08:34:52 - <Info> - Using BPF 'not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)' on iface 'eth0'
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Failed to attach filter: Cannot allocate memory
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Set AF_PACKET bpf filter "not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" failed.

... etc ...

The cause is apparently because BPF optimistion is disabled for af_packet.

// src/source-af-packet.c line 2084
    if (pcap_compile_nopcap(default_packet_size,  /* snaplen_arg */
                ptv->datalink,    /* linktype_arg */
                &filter,       /* program */
                ptv->bpf_filter, /* const char *buf */
                0,             /* optimize   <<--- HERE */
                0              /* mask */
                ) == -1) {

Testing using a little C program, unoptimised the BPF is 1104 instruction, optimised is 157 instructions. I think the kernel complains at 1024 instructions.

The "looping until out of memory" is probably some error in the recovery code, but I didn't look into that.