Bug #5633
closedPass rules on 6.0.8 are generating alert events when passing tunneled traffic
Description
I tested upgrading to suricata 6.0.8 from 6.0.6 and was suddenly being spammed with alerts on custom pass rules that should not be generating alerts at all. These are pass rules old/new that have been in use since Suricata 3.x and I have never witnessed suricata ever alerting from a pass rule signature. One of the strangest things I have ever seen.
I tried many times to reproduce this in lab with pcap samples of traffic from production that was generating the alerts, but had no success.
I finally realized that all the pass rule alerts were within a GRE or IPv6-Frag tunnel as reported by the tunnel.proto field of the json alerts.
Once I collected a production pcap sample on the tunnel IPs instead of direct src/dest I was immediately able to reproduce the issue in lab. Apparently we have some of the same types of traffic inside a tunnel as well as not.
I personally use a custom compiled Suricata 6.0.8 on CentOS 7, but I have a coworker that uses the CentOS 7 rpm package as well. His sensors are on completely different networks with completely different pass rules and he saw the exact same flood of pass rule alerts being generated when he tested. Every one of his pass rule alerts were also for traffic inside a tunnel. Neither of us changed any config at all, and only changed the installed Suricata version.
I redacted quite a lot, but this is one the sample alerts that was generated by suricata 6.0.8 for a pass rule for traffic in a tunnel. The actual IPs represented inside the tunnel should have matched with pass rule IP variables and been ignored. The actual packet was a generic syslog message.
{
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 60000139,
"rev": 2,
"signature": "CUSTOM---",
"category": "Misc activity",
"severity": 3,
"rule": "pass udp $HOME_NET any -> $INTERNAL 514 (msg:\"CUSTOM---\"; classtype:misc-activity; sid:60000139; rev:2;)"
},
"app_proto": "failed",
"community_id": "1:---",
"dest_ip": "172.0.0.0",
"dest_port": 514,
"ether": {},
"event_type": "alert",
"flow": {
"pkts_toserver": 1,
"pkts_toclient": 0,
"bytes_toserver": 178,
"bytes_toclient": 0,
"start": "2022-11-02T22:20:14.454443+0000"
},
"flow_id": 123---,
"in_iface": "mon4",
"packet": "---",
"packet_info": {
"linktype": 12
},
"payload": "---",
"payload_printable": "---",
"proto": "UDP",
"src_ip": "172.0.0.0",
"src_port": 514,
"stream": 0,
"timestamp": "2022-11-02T22:20:14.454443+0000",
"tunnel": {
"src_ip": "156.0.0.0",
"src_port": 0,
"dest_ip": "156.0.0.0",
"dest_port": 0,
"proto": "GRE",
"depth": 1
}
}
This is Suricata version 6.0.8 RELEASE Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON PROFILING TLS TLS_GNU MAGIC RUST SIMD support: SSE_4_2 SSE_4_1 SSE_3 Atomic intrinsics: 1 2 4 8 16 byte(s) 64-bits, Little-endian architecture GCC version 4.8.5 20150623 (Red Hat 4.8.5-44), C version 199901 compiled with _FORTIFY_SOURCE=0 L1 cache line size (CLS)=64 thread local storage method: __thread compiled with LibHTP v0.5.41, linked against LibHTP v0.5.41 Suricata Configuration: AF_PACKET support: yes eBPF support: no XDP support: no PF_RING support: no NFQueue support: yes NFLOG support: no IPFW support: no Netmap support: no DAG enabled: no Napatech enabled: no WinDivert enabled: no Unix socket enabled: yes Detection enabled: yes Libmagic support: yes libnss support: yes libnspr support: yes libjansson support: yes hiredis support: no hiredis async with libevent: no Prelude support: no PCRE jit: yes LUA support: yes libluajit: no GeoIP2 support: yes Non-bundled htp: no Hyperscan support: yes Libnet support: yes liblz4 support: yes HTTP2 decompression: no Rust support: yes Rust strict mode: no Rust compiler path: /usr/local/bin/rustc Rust compiler version: rustc 1.44.1 (c7087fe00 2020-06-17) Cargo path: /usr/local/bin/cargo Cargo version: cargo 1.44.1 (88ba85757 2020-06-11) Cargo vendor: yes Python support: yes Python path: /usr/bin/python2.7 Install suricatactl: yes Install suricatasc: yes Install suricata-update: yes Profiling enabled: yes Profiling locks enabled: no Plugin support (experimental): yes Development settings: Coccinelle / spatch: no Unit tests enabled: no Debug output enabled: no Debug validation enabled: no Generic build parameters: Installation prefix: /usr/local Configuration directory: /etc/suricata/ Log directory: /var/log/suricata/ --prefix /usr/local --sysconfdir /etc --localstatedir /var --datarootdir /usr/local/share Host: x86_64-pc-linux-gnu Compiler: gcc (exec name) / g++ (real) GCC Protect enabled: no GCC march native enabled: yes GCC Profile enabled: no Position Independent Executable enabled: no CFLAGS -g -O2 -std=gnu99 -march=native -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist PCAP_CFLAGS -I/usr/local/include SECCFLAGS
Files
Updated by Victor Julien almost 2 years ago
- Related to Security #5571: ips: encapsulated packet logged as dropped, but not actually dropped added
Updated by Victor Julien almost 2 years ago
- Priority changed from Urgent to High
- Target version changed from TBD to 7.0.0-rc1
- Label Needs backport to 6.0 added
Are you able to provide a test case (pcap+rules+expected output) for our Suricata-Verify repo? https://github.com/OISF/suricata-verify
Updated by Shivani Bhardwaj almost 2 years ago
- Label deleted (
Needs backport to 6.0)
Updated by Victor Julien almost 2 years ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Jeff Lucovsky
Updated by Jeff Lucovsky almost 2 years ago
Having a PCAP would help, even if the pcap has a few packets. The pcap can be posted here or shared privately (if there's sensitive information within it)
Updated by Don Williams almost 2 years ago
- File gre-sample.pcap gre-sample.pcap added
- File pass.rules pass.rules added
- File fast.log fast.log added
- File eve.json eve.json added
There is nothing special in my traffic. It is just basic GRE tunnel traffic of many different types of data.
I downloaded this gre sample from bro and wrote a very basic pass rule and was able to duplicate it.
https://github.com/bro/bro/raw/master/testing/btest/Traces/tunnels/gre-sample.pcap
With this I was able to get Suricata alerting on the pass rule rather than actually passing the traffic.
Updated by Victor Julien almost 2 years ago
FYI this appears to be already fixed in master. Needs a SV test to confirm.
Updated by Victor Julien almost 2 years ago
- Status changed from Assigned to In Progress
- Assignee changed from Jeff Lucovsky to Victor Julien
Updated by Victor Julien almost 2 years ago
- Target version changed from 7.0.0-rc1 to 6.0.9
Confirmed to work correctly in 7.0.0-beta1 and current git master.
Updated by Victor Julien almost 2 years ago
Updated by Victor Julien almost 2 years ago
- Status changed from In Progress to Closed