Project

General

Profile

Actions

Task #2313

open

tracking: save & restore state when suricata restarts

Added by Victor Julien almost 4 years ago. Updated almost 2 years ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
high
Difficulty:
medium
Label:

Description

Much requested feature: save state at exit and restore the state when suricata starts up

Relatively easy for some things: flows, stream tracking, but hard for others: stream reassembly, etc.


Subtasks 1 (1 open0 closed)

Bug #4138: A stable flow ID for dump/restore of state as well as state synchronizationNewActions

Related issues

Related to Task #2309: SuriCon 2017 brainstormNewVictor JulienActions
Related to Task #2685: SuriCon 2018 brainstormNewVictor JulienActions
Actions #1

Updated by Victor Julien almost 4 years ago

  • Related to Task #2309: SuriCon 2017 brainstorm added
Actions #2

Updated by Victor Julien over 3 years ago

  • Tracker changed from Bug to Feature
Actions #3

Updated by Victor Julien about 3 years ago

  • Effort set to high
  • Difficulty set to medium
Actions #4

Updated by Danny Browning about 3 years ago

One approach: * For things we wish to serialize, define them in rust (e.g. https://github.com/OISF/suricata/blob/master/src/flow.h#L325) * Use rust serde (msgpack or rson) to do serialize/deserialize of the state

In attempting to define an approach for this, most methods of doing serde approaches in c/c++ do not have a compatible license, while serde, serde-msgpack, and serde-ron do have compatible licenses.

Previously this approach would not have worked, because rust support was optional, but now that rust is required, defining core structures in rust will not create problems from a compilation/usage standpoint. With serde-derive, we can easily define which attributes we want to store off. One complication is suricata specific structures, such as SC_ATOMIC_DECLARE, and how we would populate the value. For anything not wrapped with suricata specific structures, we could also sub structure it to make serde support easier. Flow could consist of a sub structure FlowSave (name TBD) that is handled with serde.

One other benefit to a more standard format is the ability to "inject" data at portions of the pipeline. For cards doing layer 4 extraction, we may be able to skip acquire and decode.

Actions #5

Updated by Danny Browning almost 3 years ago

Because of the size of this feature, it should be split into smaller, easier to accomplish pieces, such as saving flowbits.

This would provide a proof of concept implementation that could be extended to the harder pieces of suricata to serialize, as additional serialization is warranted.

Actions #6

Updated by Victor Julien almost 3 years ago

  • Status changed from New to Assigned
  • Assignee changed from Anonymous to Danny Browning

Agreed, lets make this a tracking ticket. New tickets can set to be related to this one.

Actions #7

Updated by Victor Julien almost 3 years ago

  • Related to Task #2685: SuriCon 2018 brainstorm added
Actions #8

Updated by Victor Julien about 2 years ago

  • Tracker changed from Feature to Task
  • Subject changed from save & restore state when suricata restarts to tracking: save & restore state when suricata restarts
  • Status changed from Assigned to New
  • Assignee changed from Danny Browning to OISF Dev
Actions #9

Updated by Andreas Herz almost 2 years ago

we will create an additional issue for tracking the thresholds in between suricata restarts.
It can be helpful as well for elephant flows and bypassing them again after a restart.

Actions #10

Updated by Jason Ish 11 months ago

  • Related to Bug #4138: A stable flow ID for dump/restore of state as well as state synchronization added
Actions

Also available in: Atom PDF