Project

General

Profile

Actions

Feature #3636

open

eve: configuration options to enable all, none or just a default set of outputs

Added by Jason Ish over 4 years ago. Updated 5 months ago.

Status:
Assigned
Priority:
Low
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

One issue with upgrading Suricata is that new protocols will not be logged by default as outputs aren't enabled unless they are explicitly added to the types in the eve-log configuration section. This is different than the app-layer protocols that usually are enabled by default if not found in the configuration (their are exceptions to this).

It should be easier to enable the default set of outputs without having to list them all and we should use this in our default configuration. This will mean that the recommended by default outputs stays current over upgrades.

The main issue is that YAML (or maybe any configuration format) makes this hard to express.

NOTE: The following is for discussion purposes only at this time.

Magic Keywords
Introduce magic type names like:
  • all
  • defaults

(These could be __all, __defaults, etc if we're worried about name conflicts.)

Then our default configuration file could look like:

types:
  - defaults

Which would enable all the outputs except drop, netflow, dnp3, and metadata as these are not currently enabled in the default configuration. This would also be the default set enabled if the output was missing the types field.

Then to use the defaults with netflow the following configuration could be used:

types:
  - defaults
  - netflow

and if the configuration for an output needed to be modified:

types:
  - defaults
  - dns:
      # Custom options

DNS being explicitly listed with options would override its inclusion as part of defaults.

Exclusion List

If only a few specific outputs are desired its easy to explicitly list them, however its harder to express a configuration with all or defaults except specific outputs. To address this an exclude_types field could be added:

types:
  - defaults
exclude_types:
  - smb
  - nfs

[maybe disable_types is better]

New Default Configuration
types:
  - defaults
exclude_types: []

Related issues 4 (3 open1 closed)

Related to Suricata - Feature #1993: commandline: introduce --enable-all-outputs switchAssignedOISF DevActions
Related to Suricata - Optimization #2321: yaml: clean up usage of listsAssignedJason IshActions
Related to Suricata - Feature #4782: config: add command to dump all active settingsIn ProgressJason IshActions
Has duplicate Suricata - Feature #7069: config/eve: magic toggle to enable all typesRejectedOISF DevActions
Actions #1

Updated by Jason Ish over 4 years ago

  • Priority changed from Normal to Low
  • Target version set to 6.0.0beta1
Actions #2

Updated by Jason Ish over 4 years ago

  • Related to Feature #1993: commandline: introduce --enable-all-outputs switch added
Actions #3

Updated by Jason Ish over 4 years ago

  • Subject changed from eve: optiont to enable all outputs to eve: option to enable all outputs
Actions #4

Updated by Victor Julien over 4 years ago

  • Target version changed from 6.0.0beta1 to 7.0.0-beta1
Actions #5

Updated by Victor Julien over 3 years ago

  • Assignee set to Juliana Fajardini Reichow
Actions #6

Updated by Juliana Fajardini Reichow over 3 years ago

  • Status changed from New to In Progress

Hi,

my interpretation is that this issue is to address the yaml file option, since the related issue would already address cmdline alternative. Is that correct?

Actions #7

Updated by Jason Ish over 3 years ago

Juliana Fajardini Reichow wrote in #note-6:

Hi,

my interpretation is that this issue is to address the yaml file option, since the related issue would already address cmdline alternative. Is that correct?

Thats correct. The end result would be the same I think, just where its set. The command line option is the most useful for now I think.

Actions #8

Updated by Jason Ish about 3 years ago

  • Subject changed from eve: option to enable all outputs to eve: configuration options to enable all, none or just a default set of outputs
Actions #9

Updated by Jason Ish about 3 years ago

  • Description updated (diff)
  • Status changed from In Progress to Assigned
Actions #10

Updated by Jason Ish about 3 years ago

  • Description updated (diff)
Actions #11

Updated by Jason Ish about 3 years ago

Updated the description to discuss more about the problem and possible solutions.

Actions #12

Updated by Jeff Lucovsky about 3 years ago

In the use case from the description, is the assumption being made that the configuration file from the older Suricata deployment will be maintained?

Each new release, with potentially new protocols to log, would by definition already have those protocols listed under types:.

If

types:
   - defaults

is meant to help with updated Suricata versions, but not Suricata config files, I think this could work.

Actions #13

Updated by Jason Ish about 3 years ago

Jeff Lucovsky wrote in #note-12:

In the use case from the description, is the assumption being made that the configuration file from the older Suricata deployment will be maintained?

No real assumptions. We just know that some upgrades occur without a proper conversion of the configuration file as we do our best to keep old configurations working, but this does mean that new protocols that are logged by default, in say 7.0 will never be enabled for the "lazy" upgrade path from 6.0. However, the protocol decoder enabled in this "lazy" upgrade path.

Actions #14

Updated by Victor Julien over 2 years ago

  • Target version changed from 7.0.0-beta1 to 8.0.0-beta1
Actions #15

Updated by Jason Ish almost 2 years ago

Actions #16

Updated by Victor Julien 11 months ago

  • Related to Feature #4782: config: add command to dump all active settings added
Actions #17

Updated by Juliana Fajardini Reichow 6 months ago

  • Related to Feature #7069: config/eve: magic toggle to enable all types added
Actions #18

Updated by Shivani Bhardwaj 6 months ago

  • Has duplicate Feature #7069: config/eve: magic toggle to enable all types added
Actions #19

Updated by Shivani Bhardwaj 6 months ago

  • Related to deleted (Feature #7069: config/eve: magic toggle to enable all types)
Actions #20

Updated by Victor Julien 5 months ago

  • Assignee changed from Juliana Fajardini Reichow to OISF Dev
Actions

Also available in: Atom PDF