Suricata

Indira M wrote:

Hello Suricata team,

We are integrating Suricata as a library in our VPP-based packet processing plugin. Our goal is to inspect live packets received from VPP using Suricata in library mode (not offline/pcap mode), with runmode set to "live".
Could you please clarify:

What are the recommended Suricata APIs for initializing, injecting live packets, and retrieving alerts/events when Suricata is used as a library in runmode=live?
Are there any specific initialization or threading requirements for this use case?
Is there an example or documentation for using Suricata as a library for real-time/live packet inspection (not pcap file replay)?
We have referred to examples/lib/custom/main.c, but it appears to use offline/pcap mode. We want to ensure we are using the correct APIs and patterns for live packet injection and alert retrieval.

Thank you!

1. Issues Observed

1. 1. VPP CLI Blocking
   - When using Suricata's `SCRunModeLibSpawnWorker()` API to spawn Suricata worker threads, the VPP CLI (`vppctl`) became unresponsive after enabling the IDPS plugin.
   - The main thread was blocked, preventing further CLI interaction and control.
   - The Suricata worker thread entered a wait state in `TmThreadsWaitForUnpause()`.

1. 2. Crash with TmThreadsSlotProcessPkt
   - The VPP plugin uses Suricata as a library and attempts to process packets using Suricata's internal pipeline via `TmThreadsSlotProcessPkt`.
   - This function is used in Suricata's examples (`examples/lib/custom/main.c`) to process packets through the detection engine.
   - When calling `TmThreadsSlotProcessPkt(g_vpp_tv, g_vpp_tv->tm_slots, p)` from the VPP plugin worker thread, a segmentation fault occurred.
   - The crash was traced to Suricata's flow handling and detection engine code.
   - The `ThreadVars` and `FlowLookupStruct` objects were not fully initialized as expected by Suricata's pipeline.
   - Specifically, the `dtv` (DecodeThreadVars) member of `FlowLookupStruct` was either NULL or not properly set up.
   - Suricata's flow management code (e.g., `FlowSpareSync()`) accessed `dtv` and crashed when it was missing or invalid.

What are the recommended Suricata APIs for initializing, injecting live packets, and retrieving alerts/events when Suricata is used as a library in runmode=live?

It's mostly the same as the pcap file example. The example at https://github.com/OISF/suricata/tree/main-8.0.x/examples/lib/custom uses a PCAP file as its packet source, but it's not tied to that. The packets could be coming from anywhere.

I've started a new example that allows for a live pcap interface, but it's not tied to that; again, I'm just using it as a packet source.

You can see the new example in PR, https://github.com/OISF/suricata/pull/14284, or more specifically:

https://github.com/jasonish/suricata/blob/8.0.x/lib/live/v1/examples/lib/live/main.c

One thing to keep in mind is that you will need to adapt to our threading model. I'm not familiar with VPP, so I'm not able to comment on specifics of integrating with it.

You can see the new example in PR, https://github.com/OISF/suricata/pull/14284, or more specifically:

https://github.com/jasonish/suricata/blob/8.0.x/lib/live/v1/examples/lib/live/main.c

One thing to keep in mind is that you will need to adapt to our threading model. I'm not familiar with VPP, so I'm not able to comment on specifics of integrating with it.

Thanks alot for the example. It really helps. We will try this out and get back.

Here is a brief on the VPP IDPS packet inspection goal which we are working upon:
1. The VPP binary has a plugin for each module. Likewise we have a plugin for IDPS (Intrusion Detection and Prevention System).
2. The packets received on any interface would be copied CPU in the VPP module which will be given to IDPS plugin over a call back function.
3. So each packet (pkt buffer) received on IDPS plugin, needs to be injected to Suricata (running as library for inspection). The updated proofpoint rules are downloaded from cloud server per day or per week based on the configurations/design.

My next question is that, how do we handle proofpoint ruleset updates?
What are the suricata library APIs to be used for that?

My next question is that, how do we handle proofpoint ruleset updates?
What are the suricata library APIs to be used for that?

Suricata, the engine does not itself handle ruleset updates. That is typically done with external tools. Suricata includes a program, "suricata-update" that can download new rules, some users do this themselves, and other management systems probably have their own method.

The Suricata engine does however need to be told to re-read these rules when they change on disk. For now, the normal mechanisms for rule reloading are supported with no special hooks for library mode. See https://docs.suricata.io/en/suricata-8.0.2/rule-management/rule-reload.html.

Jason Ish wrote in #note-5:

The Suricata engine does however need to be told to re-read these rules when they change on disk. For now, the normal mechanisms for rule reloading are supported with no special hooks for library mode. See https://docs.suricata.io/en/suricata-8.0.2/rule-management/rule-reload.html.

Thanks for the response and the live mode example.

Could you please extend the runmode=live example to demonstrate how to re-read or reload the ruleset in non-blocking mode, suitable for use in a library context?
Specifically, we are interested in how to trigger rule reloads (e.g., after updating proofpoint.rules) without blocking packet processing or causing downtime.

An example or recommended API usage for non-blocking rule reload in runmode=live would be very helpful.