Project

General

Profile

Actions

Optimization #1418

closed

lockless flow handling during capture (autofp)

Added by Victor Julien about 9 years ago. Updated almost 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

From IRC:

<poona> Autofp only requires the flow for the packet from the lookup phase.  Maybe we can instead split it into 2 parts, the first small bodied flow lock phase in the capture thread part and the biffer  HandlePacketUpdate() post autofp under stream thread
<VictorJ> yeah I think that could work
<VictorJ> the packet taking a flow reference could even be done under the flowbucket lock (which we need in all cases), so w/o the flow lock itself
<poona> yeah
<poona> looks like the capture part doesn't need any lock at all
<VictorJ> then the capture thread can use autofp_tmqh_flow_qid_sc_atomic__ to do the flow balancing, the stream/detect thread does the (FLow)HandlePacketUpdate()
<poona> yeah
<VictorJ> cool, I like this
<VictorJ> with it, I want to do something else: move UDP app layer out of capture threads as well
<VictorJ> currently that runs in the DecodeUDP module

Steps would be:
1. create a "Flow" thread module and add it to the runmodes
2. Move most of (Flow)HandleUpdatePacket there (counters, flag updates)
3. Make packet take flow reference under FB lock, not F lock
4. Move AppLayerUDP to this new Flow thread module (out of DecodeUDP)

Need to carefully consider how this affects:
1. new flow creation
2. flow tcp reuse handling
3. locking in general

Actions #1

Updated by Victor Julien almost 9 years ago

  • Target version changed from 2.1beta4 to 3.0RC1
Actions #2

Updated by Victor Julien over 8 years ago

  • Target version changed from 3.0RC1 to 70
Actions #3

Updated by Victor Julien almost 8 years ago

  • Status changed from Assigned to Closed
  • Target version changed from 70 to 3.1rc1

This has been dealt with in a different way. The flow handling has almost completely moved out of the capture threads. The only thing left is packet decoding with flow hash calculation (for the load balancing). https://github.com/inliniac/suricata/pull/2089

Actions

Also available in: Atom PDF