Bug #6623
openSuricata BPF filter differs from tcpdump (tcpdump behaviour seems correct)
Description
Attempting to create a filter that doesn't inspect local/east-west traffic can be done various ways in tcpdump, but those same equations don't work in suricata.
In disassembling the output, I see that tcpdump: * uses negative offsets to extract bpf extensions from the kernel, but suricata's bpf doesn't (https://docs.kernel.org/networking/filter.html) * the tcpdump filter returns #262144 for true, while suricata returns #1522 (I can't find details on whether these reference specific enumerations or if anything !0 is equal).
Both binaries on this system are linked to the same libpcap which is used to compile the expression, so I don't understand why the compiled byte code differs.
With suricata, the filter either blocks everything, or blocks nothing.
As an example:
```
(((vlan) and not ((dst net 172.16.0.0/12 or 10.0.0.0/8 or 192.168.0.0/16) and (src net 172.16.0.0/12 or 10.0.0.0/8 or 192.168.0.0/16))) or ((ip) and not ((dst net 172.16.0.0/12 or 10.0.0.0/8 or 192.168.0.0/16) and (src net 172.16.0.0/12 or 10.0.0.0/8 or 192.168.0.0/16))))
```
Suricata compiles this as:
```
5/12/2023 -- 01:39:48 - <Info> - (000) ldh [12]
5/12/2023 -- 01:39:48 - <Info> - (001) jeq #0x8100 jt 3 jf 2
5/12/2023 -- 01:39:48 - <Info> - (002) jeq #0x9100 jt 3 jf 43
5/12/2023 -- 01:39:48 - <Info> - (003) ldh [16]
5/12/2023 -- 01:39:48 - <Info> - (004) jeq #0x800 jt 5 jf 23
5/12/2023 -- 01:39:48 - <Info> - (005) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (006) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (007) jeq #0xac100000 jt 14 jf 8
5/12/2023 -- 01:39:48 - <Info> - (008) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (009) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (010) jeq #0xa000000 jt 14 jf 11
5/12/2023 -- 01:39:48 - <Info> - (011) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (012) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (013) jeq #0xc0a80000 jt 14 jf 63
5/12/2023 -- 01:39:48 - <Info> - (014) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (015) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (016) jeq #0xac100000 jt 45 jf 17
5/12/2023 -- 01:39:48 - <Info> - (017) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (018) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (019) jeq #0xa000000 jt 45 jf 20
5/12/2023 -- 01:39:48 - <Info> - (020) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (021) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (022) jeq #0xc0a80000 jt 45 jf 63
5/12/2023 -- 01:39:48 - <Info> - (023) jeq #0x806 jt 25 jf 24
5/12/2023 -- 01:39:48 - <Info> - (024) jeq #0x8035 jt 25 jf 63
5/12/2023 -- 01:39:48 - <Info> - (025) ld [42]
5/12/2023 -- 01:39:48 - <Info> - (026) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (027) jeq #0xac100000 jt 34 jf 28
5/12/2023 -- 01:39:48 - <Info> - (028) ld [42]
5/12/2023 -- 01:39:48 - <Info> - (029) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (030) jeq #0xa000000 jt 34 jf 31
5/12/2023 -- 01:39:48 - <Info> - (031) ld [42]
5/12/2023 -- 01:39:48 - <Info> - (032) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (033) jeq #0xc0a80000 jt 34 jf 63
5/12/2023 -- 01:39:48 - <Info> - (034) ld [32]
5/12/2023 -- 01:39:48 - <Info> - (035) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (036) jeq #0xac100000 jt 64 jf 37
5/12/2023 -- 01:39:48 - <Info> - (037) ld [32]
5/12/2023 -- 01:39:48 - <Info> - (038) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (039) jeq #0xa000000 jt 64 jf 40
5/12/2023 -- 01:39:48 - <Info> - (040) ld [32]
5/12/2023 -- 01:39:48 - <Info> - (041) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (042) jeq #0xc0a80000 jt 64 jf 63
5/12/2023 -- 01:39:48 - <Info> - (043) ldh [16]
5/12/2023 -- 01:39:48 - <Info> - (044) jeq #0x800 jt 45 jf 64
5/12/2023 -- 01:39:48 - <Info> - (045) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (046) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (047) jeq #0xac100000 jt 54 jf 48
5/12/2023 -- 01:39:48 - <Info> - (048) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (049) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (050) jeq #0xa000000 jt 54 jf 51
5/12/2023 -- 01:39:48 - <Info> - (051) ld [34]
5/12/2023 -- 01:39:48 - <Info> - (052) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (053) jeq #0xc0a80000 jt 54 jf 63
5/12/2023 -- 01:39:48 - <Info> - (054) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (055) and #0xfff00000
5/12/2023 -- 01:39:48 - <Info> - (056) jeq #0xac100000 jt 64 jf 57
5/12/2023 -- 01:39:48 - <Info> - (057) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (058) and #0xff000000
5/12/2023 -- 01:39:48 - <Info> - (059) jeq #0xa000000 jt 64 jf 60
5/12/2023 -- 01:39:48 - <Info> - (060) ld [30]
5/12/2023 -- 01:39:48 - <Info> - (061) and #0xffff0000
5/12/2023 -- 01:39:48 - <Info> - (062) jeq #0xc0a80000 jt 64 jf 63
5/12/2023 -- 01:39:48 - <Info> - (063) ret #1522
5/12/2023 -- 01:39:48 - <Info> - (064) ret #6622
```
While tcpdump compiles it as:
```
(000) ldb [-4048]
(001) jeq #0x1 jt 2 jf 42
(002) ldh [12]
(003) jeq #0x800 jt 4 jf 22
(004) ld [30]
(005) and #0xfff00000
(006) jeq #0xac100000 jt 13 jf 7
(007) ld [30]
(008) and #0xff000000
(009) jeq #0xa000000 jt 13 jf 10
(010) ld [30]
(011) and #0xffff0000
(012) jeq #0xc0a80000 jt 13 jf 62
(013) ld [26]
(014) and #0xfff00000
(015) jeq #0xac100000 jt 44 jf 16
(016) ld [26]
(017) and #0xff000000
(018) jeq #0xa000000 jt 44 jf 19
(019) ld [26]
(020) and #0xffff0000
(021) jeq #0xc0a80000 jt 44 jf 62
(022) jeq #0x806 jt 24 jf 23
(023) jeq #0x8035 jt 24 jf 62
(024) ld [38]
(025) and #0xfff00000
(026) jeq #0xac100000 jt 33 jf 27
(027) ld [38]
(028) and #0xff000000
(029) jeq #0xa000000 jt 33 jf 30
(030) ld [38]
(031) and #0xffff0000
(032) jeq #0xc0a80000 jt 33 jf 62
(033) ld [28]
(034) and #0xfff00000
(035) jeq #0xac100000 jt 63 jf 36
(036) ld [28]
(037) and #0xff000000
(038) jeq #0xa000000 jt 63 jf 39
(039) ld [28]
(040) and #0xffff0000
(041) jeq #0xc0a80000 jt 63 jf 62
(042) ldh [12]
(043) jeq #0x800 jt 44 jf 63
(044) ld [30]
(045) and #0xfff00000
(046) jeq #0xac100000 jt 53 jf 47
(047) ld [30]
(048) and #0xff000000
(049) jeq #0xa000000 jt 53 jf 50
(050) ld [30]
(051) and #0xffff0000
(052) jeq #0xc0a80000 jt 53 jf 62
(053) ld [26]
(054) and #0xfff00000
(055) jeq #0xac100000 jt 63 jf 56
(056) ld [26]
(057) and #0xff000000
(058) jeq #0xa000000 jt 63 jf 59
(059) ld [26]
(060) and #0xffff0000
(061) jeq #0xc0a80000 jt 63 jf 62
(062) ret #262144
(063) ret #0
```
Updated by Victor Julien 11 months ago
Suricata calls pcap_compile with optimize=1 and mask=0. Are you seeing the same for tcpdump? Are you also seeing the same datalink used?
Updated by Jeff Weeks 11 months ago
Victor Julien wrote in #note-1:
Suricata calls
pcap_compilewithoptimize=1andmask=0. Are you seeing the same for tcpdump? Are you also seeing the same datalink used?
The tcpdump source, by default, uses optimize=1 and mask=0, yes (I confirmed via gdb that, upon calling pcap_compile, rcx = 1, and r8 = 0 (args 4, 5 on my amd64 platform)
The tcpdump output doesn't seem to change from tcpdump -d "..." vs tcpdump -n lan0 -d "..."
Updated by Victor Julien 10 months ago
Have you been able to make any progress on this?
It seems a possible difference could be snaplen? The 1522 number above may correlate to what Suricata sets as max packet size.
The 262144 from tcpdump looks very familiar too, in pcap file snaplens. Perhaps try setting tcpdump -s1522 ... as a test?
Updated by Victor Julien 10 months ago
Only a minor difference between those it seems:
--- /tmp/nosnap.bpf 2024-02-09 15:28:19.547489823 +0100
+++ /tmp/snap1522.bpf 2024-02-09 15:28:30.467504341 +0100
@@ -89,5 +89,5 @@
(088) ld [x + 26]
(089) and #0xffff0000
(090) jeq #0xc0a80000 jt 92 jf 91
-(091) ret #262144
+(091) ret #1522
(092) ret #0
Updated by Victor Julien 9 months ago
Hmm another thought: af-packet actually gets packets with the vlan header stripped from it. I don't know how this might affect things. The bpf program is set on the af-packet socket, but I don't know how it is evaluated wrt vlan.