Project

General

Profile

Actions

Bug #1780

closed

VLAN tags not forwarded in afpacket inline mode

Added by Alexander Gozman over 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
High
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

It seems that VLAN tags are not forwarded when suricata runs in afpacket inline mode: incoming packets contain VLAN tag, but outgoing do not. The test was performed on 2.1beta4, but I suppose that the problem remains in more recent versions.
Here's the excerpt from suricata.yaml:

vlan:
  use-for-tracking: true
runmode: workers
af-packet:
- interface: eth1
  copy-iface: eth5
  cluster-id: 3
  cluster-type: cluster_flow
  use-mmap: yes
  copy-mode: ips
  threads: 4

- interface: eth5
  copy-iface: eth1
  cluster-id: 4
  cluster-type: cluster_flow
  use-mmap: yes
  copy-mode: ips
  threads: 4

All offloading features that may affect seem to be turned off. First interface:

# ethtool -k eth1
Features for eth1:
rx-checksumming: on
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: on
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

And the second one:

# ethtool -k eth5
Features for eth5:
rx-checksumming: on
tx-checksumming: on
    tx-checksum-ipv4: on
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: on
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

Can send PCAPs privately if needed.


Related issues

Related to Bug #2934: VLAN tags stripped when saving pcap logNewOISF DevActions
Related to Bug #2478: PCAP logging does not include 802.1q header when using af-packetFeedbackPaul BakoyiannisActions
Actions #1

Updated by Victor Julien over 5 years ago

  • Status changed from New to Assigned
  • Assignee set to Eric Leblond
  • Target version set to 70
Actions #2

Updated by Alexander Gozman over 5 years ago

The problem occurs on kernel 3.14.43. Probably the problem is that sll_protocol field is not set in AFPWritePacket and kernel just ignores VLAN info, but it's just a supposition.

Actions #3

Updated by Alexander Gozman over 5 years ago

Addition: both network cards use igb drivers (i82576). Maybe it's a driver issue... We also try to understand what's going on.

Actions #4

Updated by Alexander Gozman over 5 years ago

The problem can be reproduced only on real network traffic. Locally tcpreplayed packets perfectly pass through suricata without VLAN truncation. We need to go deeper...

Actions #5

Updated by Alexander Gozman over 5 years ago

It seems that the problem is in how suricata works in afpacket mode. We've done some research about it and got the following result:
  1. AFRead() ignores VLAN info - no auxilary data is processed.
  2. AFPReadFromRing() gets VLAN info from auxilary data.
  3. AFPWritePacket() sends plain ethernet packet without [auxilary] VLAN data. If we create a custom packet in AFPWritePacket() and put VLAN info there, everything is passed without any problems. For example (quick'n'dirty, just to test the idea):
if (p->vlan_idx == 1) {
    size_t offset = ETH_ALEN * 2;
    char buffer[1600] = {0};

    memcpy(buffer, GET_PKT_DATA(p), GET_PKT_LEN(p));
    memmove(buffer + offset + 4, buffer + offset, GET_PKT_LEN(p)-offset);

    uint16_t val = htons(0x8100);
    memcpy(buffer + offset, &val, 2);
    val = htons(p->vlan_id[0]);
    memcpy(buffer + offset + 2, &val, 2);
    sendto(socket, buffer, GET_PKT_LEN(p) + 4, 0, (struct sockaddr*) &socket_address, sizeof(struct sockaddr_ll));
} else {
    ... // Send as usual
}

Hope you find this useful.

Actions #6

Updated by Victor Julien over 5 years ago

Thanks for the analysis. Does this mean that if you force the code to use AFPRead() it works correctly?

Actions #7

Updated by Alexander Gozman over 5 years ago

We tried to use AFPRead(), but it didn't help. Actually, this function is called only when 'use-mmap' is set to 'no'. And it only gets data from socket, while AFPReadFromRing() also reads a auxilary data like VLAN id.

So we've concluded that there're two buggy functions:
- AFPRead: it ignores auxilary data. It's bad, but not critical - we have AFPReadFromRing.
- AFPWritePacket(): it always writes plain ethernet, no 802.1q or smth. And this is a real problem.

Actions #8

Updated by Victor Julien over 5 years ago

The problem is that the kernel always strips the vlan header from the data passed to Suricata. Kernel folks suggested we forge a vlan header and add it to the out going packet. Quite ugly :(

Actions #9

Updated by Eric Leblond almost 5 years ago

I've got an implementation here that seems to work https://github.com/inliniac/suricata/pull/2404

Actions #10

Updated by Victor Julien over 4 years ago

  • Status changed from Assigned to Closed
  • Target version changed from 70 to 3.2.1
Actions #11

Updated by Victor Julien over 2 years ago

  • Related to Bug #2934: VLAN tags stripped when saving pcap log added
Actions #12

Updated by Andreas Herz about 2 years ago

  • Related to Bug #2478: PCAP logging does not include 802.1q header when using af-packet added
Actions

Also available in: Atom PDF