Bug #1780
closedVLAN tags not forwarded in afpacket inline mode
Description
It seems that VLAN tags are not forwarded when suricata runs in afpacket inline mode: incoming packets contain VLAN tag, but outgoing do not. The test was performed on 2.1beta4, but I suppose that the problem remains in more recent versions.
Here's the excerpt from suricata.yaml:
vlan: use-for-tracking: true runmode: workers af-packet: - interface: eth1 copy-iface: eth5 cluster-id: 3 cluster-type: cluster_flow use-mmap: yes copy-mode: ips threads: 4 - interface: eth5 copy-iface: eth1 cluster-id: 4 cluster-type: cluster_flow use-mmap: yes copy-mode: ips threads: 4
All offloading features that may affect seem to be turned off. First interface:
# ethtool -k eth1
Features for eth1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
And the second one:
# ethtool -k eth5
Features for eth5:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
Can send PCAPs privately if needed.
Updated by Victor Julien almost 8 years ago
- Status changed from New to Assigned
- Assignee set to Eric Leblond
- Target version set to 70
Updated by Alexander Gozman almost 8 years ago
The problem occurs on kernel 3.14.43. Probably the problem is that sll_protocol field is not set in AFPWritePacket and kernel just ignores VLAN info, but it's just a supposition.
Updated by Alexander Gozman almost 8 years ago
Addition: both network cards use igb drivers (i82576). Maybe it's a driver issue... We also try to understand what's going on.
Updated by Alexander Gozman almost 8 years ago
The problem can be reproduced only on real network traffic. Locally tcpreplayed packets perfectly pass through suricata without VLAN truncation. We need to go deeper...
Updated by Alexander Gozman almost 8 years ago
- AFRead() ignores VLAN info - no auxilary data is processed.
- AFPReadFromRing() gets VLAN info from auxilary data.
- AFPWritePacket() sends plain ethernet packet without [auxilary] VLAN data. If we create a custom packet in AFPWritePacket() and put VLAN info there, everything is passed without any problems. For example (quick'n'dirty, just to test the idea):
if (p->vlan_idx == 1) {
size_t offset = ETH_ALEN * 2;
char buffer[1600] = {0};
memcpy(buffer, GET_PKT_DATA(p), GET_PKT_LEN(p));
memmove(buffer + offset + 4, buffer + offset, GET_PKT_LEN(p)-offset);
uint16_t val = htons(0x8100);
memcpy(buffer + offset, &val, 2);
val = htons(p->vlan_id[0]);
memcpy(buffer + offset + 2, &val, 2);
sendto(socket, buffer, GET_PKT_LEN(p) + 4, 0, (struct sockaddr*) &socket_address, sizeof(struct sockaddr_ll));
} else {
... // Send as usual
}
Hope you find this useful.
Updated by Victor Julien almost 8 years ago
Thanks for the analysis. Does this mean that if you force the code to use AFPRead() it works correctly?
Updated by Alexander Gozman almost 8 years ago
We tried to use AFPRead(), but it didn't help. Actually, this function is called only when 'use-mmap' is set to 'no'. And it only gets data from socket, while AFPReadFromRing() also reads a auxilary data like VLAN id.
So we've concluded that there're two buggy functions:
- AFPRead: it ignores auxilary data. It's bad, but not critical - we have AFPReadFromRing.
- AFPWritePacket(): it always writes plain ethernet, no 802.1q or smth. And this is a real problem.
Updated by Victor Julien almost 8 years ago
The problem is that the kernel always strips the vlan header from the data passed to Suricata. Kernel folks suggested we forge a vlan header and add it to the out going packet. Quite ugly :(
Updated by Eric Leblond over 7 years ago
I've got an implementation here that seems to work https://github.com/inliniac/suricata/pull/2404
Updated by Victor Julien about 7 years ago
- Status changed from Assigned to Closed
- Target version changed from 70 to 3.2.1
Updated by Victor Julien about 5 years ago
- Related to Bug #2934: VLAN tags stripped when saving pcap log added
Updated by Andreas Herz over 4 years ago
- Related to Bug #2478: PCAP logging does not include 802.1q header when using af-packet added