Support #2564
Updated by Victor Julien over 5 years ago
I am using Suricata in a somewhat unusual fashion; I mostly just want to run a lua script against the content of each HTTP POST that leaves our network. The server this is all running on is tapped into a multi-gig fibre link, but all of the initial testing done was over the (copper, if it matters) management link on Ubuntu (16.04) on a Cisco UCS server. The only rules file enabled is a rule with a single alert (as below) in order to reduce load
I modified the YAML file to enable lua in the output (as described in http://suricata.readthedocs.io/en/latest/output/lua-output.html). The lua script itself I can sanitize and post if required, but I think the problem is occurring before that point (And is quite possibly based on my misunderstanding of some configuration option somewhere).
The alert that I am having trouble with seems straightforward: alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg: "hit"; content:"POST"; http_method;)
This alert works over the management interface (Triggered using lynx in another SSH session), and causes the lua script I wrote to execute, inspect the POST, etc. However, in trying to move this to the live traffic, nothing worked; Same rules file, same lua script, same command line invocation but with a different interface, and the alert never seemed to trip (As confirmed by tailing fast.log).
Much experimentation later, I have some idea what the problem may be: The above rule with just a "content:"POST"" (no http_method) triggers many times a second on live traffic (as it should), but nothing I put into "content" triggers if it is looking in the http_method modifier (Including GET, which would be triggering thousands of times a second). However, the lua script, (presumably because of returning a needs["protocol"] = "http"?), never makes it to the log function, only init/setup. I enabled the pcap mode, fired up suricata for a few moments, and then disabled it. I downloaded the pcap to my workstation and wireshark cheerfully opens it, shows http streams including POSTs, etc. (It would be difficult, but I guess not impossible, to sanitize a pcap snippet enough to be able to provide it, as this is live traffic from an educational institution). It seems that the alert is never seeing the http_method?
I'm not certain why this interface doesn't seem to have this trip; I'm more programming-orientated than networking, however, so I may be missing something here: The recommendations for a bug filed indicate that build-info may be useful, so I included that at the bottom; I'm happy to retrieve any other information that may be useful as well on request: Immediately following is an attempt to be pro-active with some information
The traffic that isn't triggering comes from one of our multiple internet links, through an optical tap fed into an SFP in the Cisco UCS. Is there something I am missing in setup to tell suricata that this is tapped, not inline traffic, or fibre vs copper?
lshw for the relevant adapters:
*-network:0
description: Ethernet interface
product: I350 Gigabit Network Connection
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:01:00.0
logical name: enp1s0f0
version: 01
serial: 28:6f:7f:31:9e:40
size: 1Gbit/s
capacity: 1Gbit/s
width: 32 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=igb driverversion=5.3.0-k duplex=full firmware=1.63, 0x80000b15, 0.384.130 ip=<redacted> latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
resources: irq:24 memory:c7000000-c70fffff ioport:6020(size=32) memory:c7204000-c7207fff memory:c7180000-c71fffff memory:3bfffe60000-3bfffe7ffff memory:3bfffe40000-3bfffe5ffff
*-network
description: Ethernet interface
product: VIC Ethernet NIC
vendor: Cisco Systems Inc
physical id: 0
bus info: pci@0000:0a:00.0
logical name: enp10s0
version: a2
serial: 00:a6:ca:e6:ef:42
size: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: msi msix pm pciexpress bus_master cap_list ethernet physical fibre
configuration: autonegotiation=off broadcast=yes driver=enic driverversion=2.3.0.20 duplex=full firmware=4.1(2d) latency=0 link=yes multicast=yes port=fibre speed=10Gbit/s
resources: irq:46 memory:c6b00000-c6b07fff memory:c6b08000-c6b09fff
Command line used for invocation:
sudo suricata -s /home/<dir>/signature.rules --af-packet=enp10s0
(I have also used just -i for the interface, and tried tweaking some settings in the AF_Packet portion of the YAML file, and based off some internet research, tried --set vlan.use-for-tracking=false in case that was an issue (But I believe there is no VLAN tagging on this link at this point of intercept))
This behavior is in 4.0.5 release, but I was previously using 4.0.0 and had the same troubles (I updated to check if this was an easy solve)
--build-info
<pre>
This is Suricata version 4.0.5 RELEASE
Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS MAGIC
SIMD support: none
Atomic intrisics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 5.4.0 20160609, C version 199901
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.27, linked against LibHTP v0.5.27
Suricata Configuration:
AF_PACKET support: yes
PF_RING support: no
NFQueue support: yes
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: yes
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes, through luajit
libluajit: yes
libgeoip: yes
Non-bundled htp: yes
Old barnyard2 support: no
CUDA enabled: no
Hyperscan support: no
Libnet support: yes
Rust support (experimental): no
Experimental Rust parsers: no
Rust strict mode: no
Suricatasc install: yes
Profiling enabled: no
Profiling locks enabled: no
Development settings:
Coccinelle / spatch: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Generic build parameters:
Installation prefix: /usr
Configuration directory: /etc/suricata/
Log directory: /var/log/suricata/
--prefix /usr
--sysconfdir /etc
--localstatedir /var
Host: x86_64-pc-linux-gnu
Compiler: gcc (exec name) / gcc (real)
GCC Protect enabled: yes
GCC march native enabled: no
GCC Profile enabled: no
Position Independent Executable enabled: yes
CFLAGS -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security
PCAP_CFLAGS -I/usr/include
SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
</pre>