Feature #1661


Improved support for xbits/hostbits (in particular ip_pair) when running with multiple threads

Added by Chris Wakelin about 8 years ago. Updated 12 months ago.

Target version:


I have been trying the the new xbit/hostbit features. I've put some test rules in experimental.rules

The problem is that when using autofp or workers mode there's no guarantee the relevant traffic will all be in the same thread and hence match the xbits. For example the Neutrino exploit kit pcap in will hit the "ET EXPERIMENTAL (data) download with high entropy after plugin file download" rule for the RC4-encrypted Flash payload with runmode=single but not for runmode=autofp :-

alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET EXPERIMENTAL Flash file download (CWS)"; flow:established,from_server; file_data; content:"CWS"; within:3; xbits:set,ET.pluginfile,track ip_pair, expire 120; flowbits:noalert; classtype:bad-unknown; sid:380000001; rev:1;)

alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET EXPERIMENTAL (data) download with high entropy after plugin file download"; flow:from_server,established; filesize:>10240; filemagic:"data"; filemagic:!" data"; xbits:isset,ET.pluginfile,track ip_pair; luajit:suri-high-entropy.lua; classtype:bad-unknown; sid:380000015; rev:1;)

One possibility is to use packet-schedulers that hash on IP pair rather than 5-tuple; e.g. for PF_RING it would be possible to use libzero and a custom distribution function.

I created a patch to provide an "ippair" scheduler for autofp (attached), but I'm not sure it's a sensible thing in practice (except perhaps in sandboxes), the downside of course being that traffic may be very unevenly distributed between threads.

I think Victor mentioned that there may be some improvements to be had by prioritising IP pair checks in the engine. Are there other suggestions to make this feature more usable?


autofp_ippair.patch (2.67 KB) autofp_ippair.patch Chris Wakelin, 01/05/2016 10:10 AM
Actions #1

Updated by Andreas Herz about 8 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD
Actions #2

Updated by Victor Julien about 8 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Chris Wakelin

I don't mind taking in such a patch. Can you submit it through the proper channel as documented here?

Actions #3

Updated by Chris Wakelin almost 8 years ago

Finally got around to doing this -

Actions #4

Updated by Victor Julien almost 8 years ago

  • Status changed from Assigned to Closed
  • Target version changed from TBD to 3.0.1

Also available in: Atom PDF