Project

General

Profile

Actions

Bug #2170

closed

Suricata fails on large BPFs with AF_PACKET

Added by Martijn van Oosterhout about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

When starting suricata with a very large BPF, it fails and starts a loop which causes Suricata to eventually use all memory.

The configuration I used is:

af-packet:
  - interface: "eth0" 
    cluster-id: 2
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    threads: 1
    ring-size: 609144
    bpf-filter: "not (host 192.168.0.1 and portrange 8000-8001)  
    and not (host 192.168.20.3 and portrange 8000-8001)  
    and not (host 192.168.22.15 and port (4001 or 4002 or 4008))  
    and not (host 192.168.22.15 and port (4001 or 4002 or 4008))  
    and not (host 192.168.22.1 and port 443)  
    and not (host 192.168.20.3 and port 902)  
    and not (host 192.168.20.7 and port 902)  
    and not (host 192.168.20.8 and port 902)  

    and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" 

The output from Suricata is as follows:

30/6/2017 -- 08:34:51 - <Info> - Including configuration file /srv/suricata/bond0/suricata.yaml.
30/6/2017 -- 08:34:51 - <Notice> - This is Suricata version 3.2.2 RELEASE
30/6/2017 -- 08:34:51 - <Info> - CPUs/cores online: 1
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DNS_CONFIG(240)] - no DNS UDP config found, enabling DNS detection on port 53.
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DNS_CONFIG(240)] - no DNS TCP config found, enabling DNS detection on port 53.
30/6/2017 -- 08:34:51 - <Info> - No 'host-mode': suricata is in IDS mode, using default setting 'sniffer-only'
30/6/2017 -- 08:34:51 - <Info> - 1 rule files processed. 4 rules successfully loaded, 0 rules failed
30/6/2017 -- 08:34:51 - <Info> - 4 signatures processed. 0 are IP-only rules, 4 are inspecting packet payload, 0 inspect application layer, 0 are decoder event only
30/6/2017 -- 08:34:51 - <Info> - Threshold config parsed: 0 rule(s) found
30/6/2017 -- 08:34:51 - <Info> - dropped the caps for main thread
30/6/2017 -- 08:34:51 - <Info> - unified2-alert "limit" value of 1 assumed to be pre-1.2 style: setting limit to 1mb
30/6/2017 -- 08:34:51 - <Info> - Unified2-alert initialized: filename unified2.alert, limit 1 MB
30/6/2017 -- 08:34:51 - <Info> - eve-log output device (unix_dgram) initialized: /var/run/suricata/bond0_output.sock
30/6/2017 -- 08:34:51 - <Warning> - [ERRCODE: SC_ERR_DEPRECATED_CONF(274)] - deprecated 'force-md5' option found. Please use 'force-hash: [md5]' instead
30/6/2017 -- 08:34:51 - <Info> - forcing md5 calculation for logged files
30/6/2017 -- 08:34:51 - <Info> - Going to use 1 thread(s)
30/6/2017 -- 08:34:51 - <Info> - Using unix socket file '/var/run/suricata//bond0'
30/6/2017 -- 08:34:51 - <Notice> - all 1 packet processing threads, 4 management threads initialized, engine started.
30/6/2017 -- 08:34:52 - <Info> - Using BPF 'not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)' on iface 'eth0'
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Failed to attach filter: Cannot allocate memory
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Set AF_PACKET bpf filter "not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" failed.
30/6/2017 -- 08:34:52 - <Info> - All AFP capture threads are running.
30/6/2017 -- 08:34:52 - <Info> - Using BPF 'not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)' on iface 'eth0'
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Failed to attach filter: Cannot allocate memory
30/6/2017 -- 08:34:52 - <Error> - [ERRCODE: SC_ERR_AFP_CREATE(190)] - Set AF_PACKET bpf filter "not (host 192.168.0.1 and portrange 8000-8001) and not (host 192.168.20.3 and portrange 8000-8001) and not (host 192.168.22.15 and port (4001 or 4002 or 4008)) and not (host 192.168.22.16 and port (4001 or 4002 or 4008)) and not (host 192.168.22.1 and port 443) and not (host 192.168.20.3 and port 902) and not (host 192.168.20.7 and port 902) and not (host 192.168.20.8 and port 902)
and not (host 192.168.24.6 and net 192.168.72.0/24 and port 4008)" failed.

... etc ...

The cause is apparently because BPF optimistion is disabled for af_packet.

// src/source-af-packet.c line 2084
    if (pcap_compile_nopcap(default_packet_size,  /* snaplen_arg */
                ptv->datalink,    /* linktype_arg */
                &filter,       /* program */
                ptv->bpf_filter, /* const char *buf */
                0,             /* optimize   <<--- HERE */
                0              /* mask */
                ) == -1) {

Testing using a little C program, unoptimised the BPF is 1104 instruction, optimised is 157 instructions. I think the kernel complains at 1024 instructions.

The "looping until out of memory" is probably some error in the recovery code, but I didn't look into that.

Actions #1

Updated by Eric Leblond about 5 years ago

Good catch. Do you wanna submit a PR on github or do you want me to do it ?

Actions #2

Updated by Martijn van Oosterhout about 5 years ago

The CLA is likely to be an issue, so if you can do it that would be great. Thanks.

Maybe also backport (don't know the policy)?

Actions #3

Updated by Martijn van Oosterhout about 5 years ago

Incidentally, the reason for running out of memory is because in the error case (in various places) it frees the frame list:

    if (ptv->ring_v2)
        SCFree(ptv->ring_v2);
    if (ptv->ring_v3)
        SCFree(ptv->ring_v3);

But doesn't call munmap() to free the actual ring itself.

Actions #4

Updated by Eric Leblond about 5 years ago

Thanks a lot for the help. I'm gonna propose a PR for master and 3.2.x.

Actions #5

Updated by Andreas Herz about 5 years ago

  • Assignee set to Eric Leblond
  • Target version set to 70
Actions #6

Updated by Victor Julien about 5 years ago

  • Status changed from New to Closed
  • Target version changed from 70 to 4.0.0
Actions

Also available in: Atom PDF