Suricata

I've updated to the latest stable PF_RING 4.7.1, and Suricata git-master (plus my patch for extra fields in http.log), though I saw this problem with PF_RING 4.6.5 as well.

I have some pcaps of obfuscated Javascript badness from various drive-by download sites which should match ET "Obfuscated Javascript" rules with some of them also matching "Driveby Download Secondary Request" (usually ".php?tp=<hex-string>"). I've been sending them down a wire with tcpreplay and found (RDG are my versions of the rules, now the same with one small exception)

1) with "suricata -i eth2"

fast.log:

07/27/2011-19:05:53.912683  [**] [1:2012401:9] ET CURRENT_EVENTS Driveby Download Secondary Request [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:05:53.912683  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:05:53.912832  [**] [1:2012401:9] ET CURRENT_EVENTS Driveby Download Secondary Request [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:05:53.912832  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80

(bogus repeat, which doesn't occur with local pcap and runmode=single or autofp, I think)

07/27/2011-19:05:53.934902  [**] [1:2013313:1] ET TROJAN Obfuscated Javascript Often Used in Drivebys 3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 193.105.154.135:80 -> 134.225.xxx.xxx:60807
07/27/2011-19:05:53.934902  [**] [1:378000108:1] RDG Obfuscated javascript #3 - used in Driveby [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 193.105.154.135:80 -> 134.225.xxx.xxx:60807
07/27/2011-19:08:27.609831  [**] [1:2013237:4] ET CURRENT_EVENTS Obfuscated Javascript Often Used in Drivebys [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 75.127.110.97:80 -> 134.225.xxx.xxx:51511
07/27/2011-19:08:27.609831  [**] [1:378000106:2] RDG Obfuscated javascript - used in Driveby [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 75.127.110.97:80 -> 134.225.xxx.xxx:51511

(no tp= or similar parameter, so no "secondary request" match)

07/27/2011-19:13:57.669033  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:43138 -> 109.230.222.234:80
07/27/2011-19:13:57.686410  [**] [1:2013237:4] ET CURRENT_EVENTS Obfuscated Javascript Often Used in Drivebys [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 109.230.222.234:80 -> 134.225.xxx.xxx:43138
07/27/2011-19:13:57.686410  [**] [1:378000106:2] RDG Obfuscated javascript - used in Driveby [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 109.230.222.234:80 -> 134.225.xxx.xxx:43138

(extra "lpg" query-string parameter matches my version, not ET's; but I've actually only seen this once)

07/27/2011-19:15:53.545058  [**] [1:2013314:4] ET TROJAN Obfuscated Javascript Often Used in Drivebys 2 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 89.46.251.147:80 -> 134.225.xxx.xxx:44064
07/27/2011-19:15:53.545058  [**] [1:378000107:1] RDG Obfuscated javascript #2 - used in Driveby [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 89.46.251.147:80 -> 134.225.xxx.xxx:44064

http.log:

07/27/2011-19:05:54.032543 inimqical32.com [**] /index.php?tp=4e6c58ba0ffb9d5c [**] Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0 [**] http://gordak3.com/ [**] GET [**] HTTP/1.1 [**] 200 [**] 63726 bytes [**] 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:08:27.982885 75.127.110.97 [**] /Home/index.php [**] Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB7.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) [**] http://www.makeupgeek.com/products/mac-powder-blush-dame/ [**] GET [**] HTTP/1.1 [**] 200 [**] 46007 bytes [**] 134.225.xxx.xxx:51511 -> 75.127.110.97:80
07/27/2011-19:13:57.725699 tolias.in [**] /index.php?tp=f7eaea96cb2cd72c&lpg=latjul14 [**] Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1 [**] http://encourageyou.eu/newadv.php?cid=latprod28 [**] GET [**] HTTP/1.1 [**] 200 [**] 24415 bytes [**] 134.225.xxx.xxx:43138 -> 109.230.222.234:80
07/27/2011-19:15:53.642001 www.dybvtgld.cjb.net [**] /s5pu0gtf/?2 [**] Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0 [**] http://www.mrexcel.com/forum/register.php?a=act&u=186771&i=bbafb8ca5f12557453ab5bfb66d117cadf15c98d [**] GET [**] HTTP/1.1 [**] 200 [**] 19927 bytes [**] 134.225.xxx.xxx:44064 -> 89.46.251.147:80

(all correct!)

2) with "suricata --pfring-int=eth2" (receive-threads = 1, but I had the same behaviour before Will's patch to not set the cluster_id in this case; also runmode=autofp doesn't make any difference, I think)

fast.log:

07/27/2011-19:18:58.336010  [**] [1:2012401:9] ET CURRENT_EVENTS Driveby Download Secondary Request [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:18:58.336010  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:18:58.336160  [**] [1:2012401:9] ET CURRENT_EVENTS Driveby Download Secondary Request [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:18:58.336160  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:60807 -> 193.105.154.135:80
07/27/2011-19:28:05.509358  [**] [1:378000105:2] RDG Driveby Download Secondary Request #3 [**] [Classification: A Network Trojan was Detected] [Priority: 1] {TCP} 134.225.xxx.xxx:43138 -> 109.230.222.234:80

http.log

07/27/2011-19:28:05.725319 tolias.in [**] /index.php?tp=f7eaea96cb2cd72c&lpg=latjul14 [**] Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1 [**] http://encourageyou.eu/newadv.php?cid=latprod28 [**] GET [**] HTTP/1.1 [**]  [**] 0 bytes [**] 134.225.xxx.xxx:43138 -> 109.230.222.234:80
07/27/2011-19:29:25.587388 www.dybvtgld.cjb.net [**] /s5pu0gtf/?2 [**] Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0 [**] http://www.mrexcel.com/forum/register.php?a=act&u=186771&i=bbafb8ca5f12557453ab5bfb66d117cadf15c98d [**] GET [**] HTTP/1.1 [**]  [**] 0 bytes [**] 134.225.xxx.xxx:44064 -> 89.46.251.147:80

So the Obfuscated Javascript alerts are missing, together with their HTTP requests.

suricata.log had

[17375] 27/7/2011 -- 19:18:58 - (app-layer-parser.c:955) <Error> (AppLayerParse) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error occured in parsing "http" app layer protocol, using network protocol 6, source IP address 134.225.xxx.xxx, destination IP address 193.105.154.135, src port 60807 and dst port 80
[17375] 27/7/2011 -- 19:24:52 - (app-layer-parser.c:955) <Error> (AppLayerParse) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error occured in parsing "http" app layer protocol, using network protocol 6, source IP address 134.225.xxx.xxx, destination IP address 75.127.110.97, src port 51511 and dst port 80

which didn't occur with 1) and would have matched the first two pcaps