Project

General

Profile

Actions

Bug #2954

open

Strange interaction with afpacket - high CPU usage and no packet processing

Added by Doug Burks almost 5 years ago. Updated almost 5 years ago.

Status:
New
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Greetings from the Security Onion community!

As you may know, Security Onion uses Suricata, netsniff-ng, and Bro. We recently started defaulting our Suricata configuration to use afpacket instead of pfring. I just noticed a strange interaction with Suricata and afpacket resulting in high CPU usage and no packet processing.

Here is what I know so far:

- If Suricata is the only sniffing process running (no netsniff-ng), then it processes traffic with normal CPU usage.

- If netsniff-ng is running and Suricata starts in afpacket mode, Suricata spikes to 100% CPU usage and stays there for a long period of time. The log reports that Suricata is initialized, but it is not processing any traffic.

- If I then try to restart Suricata immediately, it again spikes to 100% CPU usage and does not process traffic.

- After waiting a certain amount of time, I can restart Suricata and it will process traffic with normal CPU usage.

- If I start Suricata first without netsniff-ng running, Suricata will process traffic with normal CPU usage. If I then start netsniff-ng, Suricata will immediately spike to 100% CPU usage.

- netsniff-ng consumes traffic via tpacket v3.

- Bro consumes traffic from afpacket and is unaffected by all of this.

- Suricata running in pfring mode is unaffected by all of this.

- Duplicated this issue on multiple different physical boxes with Intel NICs, have not been able to duplicate in a VM yet.

- Duplicated this issue with Suricata configured for 1, 2, and 3 afpacket workers.

- Ubuntu 16.04

- Linux 4.15.0-48-generic

It almost feels as if netsniff is creating some kind of lock or changing afpacket processing in such a way as to cause Suricata to spin. After a certain amount of time, that reverts and Suricata is able to process traffic properly. Bro is unaffected by all of this, so I wonder if Suricata can workaround this condition in the same way.

Thoughts?

Thanks in advance!


Files

suricata-output.txt (191 Bytes) suricata-output.txt Doug Burks, 05/02/2019 12:52 PM
suricata-yaml-afpacket.txt (4.85 KB) suricata-yaml-afpacket.txt Doug Burks, 05/02/2019 12:52 PM
suricata-cmdline.txt (143 Bytes) suricata-cmdline.txt Doug Burks, 05/02/2019 12:52 PM
netsniff-output.txt (135 Bytes) netsniff-output.txt Doug Burks, 05/02/2019 12:52 PM
netsniff-cmdline.txt (186 Bytes) netsniff-cmdline.txt Doug Burks, 05/02/2019 12:52 PM

Related issues 1 (1 open0 closed)

Related to Suricata - Feature #1954: runtime option/flag to disable hardware timestamp supportNewCommunity TicketActions
Actions

Also available in: Atom PDF