Project

General

Profile

Actions

Feature #1661

closed

Improved support for xbits/hostbits (in particular ip_pair) when running with multiple threads

Added by Chris Wakelin almost 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

I have been trying the the new xbit/hostbit features. I've put some test rules in https://github.com/EmergingThreats/et-luajit-scripts experimental.rules

The problem is that when using autofp or workers mode there's no guarantee the relevant traffic will all be in the same thread and hence match the xbits. For example the Neutrino exploit kit pcap in http://www.malware-traffic-analysis.net/2015/08/31/index.html will hit the "ET EXPERIMENTAL (data) download with high entropy after plugin file download" rule for the RC4-encrypted Flash payload with runmode=single but not for runmode=autofp :-

alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET EXPERIMENTAL Flash file download (CWS)"; flow:established,from_server; file_data; content:"CWS"; within:3; xbits:set,ET.pluginfile,track ip_pair, expire 120; flowbits:noalert; classtype:bad-unknown; sid:380000001; rev:1;)

alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET EXPERIMENTAL (data) download with high entropy after plugin file download"; flow:from_server,established; filesize:>10240; filemagic:"data"; filemagic:!" data"; xbits:isset,ET.pluginfile,track ip_pair; luajit:suri-high-entropy.lua; classtype:bad-unknown; sid:380000015; rev:1;)

One possibility is to use packet-schedulers that hash on IP pair rather than 5-tuple; e.g. for PF_RING it would be possible to use libzero and a custom distribution function.

I created a patch to provide an "ippair" scheduler for autofp (attached), but I'm not sure it's a sensible thing in practice (except perhaps in sandboxes), the downside of course being that traffic may be very unevenly distributed between threads.

I think Victor mentioned that there may be some improvements to be had by prioritising IP pair checks in the engine. Are there other suggestions to make this feature more usable?


Files

autofp_ippair.patch (2.67 KB) autofp_ippair.patch Chris Wakelin, 01/05/2016 10:10 AM
Actions #1

Updated by Andreas Herz almost 6 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD
Actions #2

Updated by Victor Julien almost 6 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Chris Wakelin

I don't mind taking in such a patch. Can you submit it through the proper channel as documented here? https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Contributing

Actions #3

Updated by Chris Wakelin over 5 years ago

Finally got around to doing this - https://github.com/inliniac/suricata/pull/1958

Actions #4

Updated by Victor Julien over 5 years ago

  • Status changed from Assigned to Closed
  • Target version changed from TBD to 3.0.1
Actions

Also available in: Atom PDF