Suricata

I can reproduce a case where af-packet threads get no more traffic despite traffic being pushed to them. The poll() call in the Suricata af-packet code times out and returns 0 and never recovers. Drop stats for the thread are increasing rapidly as no packets are read. It's as if the socket is somehow getting confused. There is no errno being set or a kernel message in dmesg.

The issue will appear consistently within half an hour of a sustained t-rex test. The test I'm using is simple:

victor@z420:/opt/trex/v2.92$ sudo ./t-rex-64 -f cap2/http_very_long.yaml -c 1 -m 11111 -d 7200 --active-flows 512 -p --cfg ../cfg_ids_10gb_x520_tr1.yaml

Suricata runs on another matchine:

./src/suricata -c ids.yaml --af-packet -l /var/log/suricata/ -S bypass.rules -vv --set flow.hash-size=1000000

af-packet config:

af-packet:
  - interface: enp65s0f1
    cluster-id: 20

  - interface: default
    threads: 16
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    mmap-locked: no
    #tpacket-v3: yes
    ring-size: 8192
    block-size: 262144

NIC stats in ethtool show that the traffic continues to be well balanced.

Kernel: Linux tr1 5.11.0-38-generic #42~20.04.1-Ubuntu SMP Tue Sep 28 20:41:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
NIC: Intel X710, i40e driver