Project

General

Profile

Actions

Optimization #2321

open

yaml: clean up usage of lists

Added by Victor Julien almost 7 years ago. Updated almost 2 years ago.

Status:
Assigned
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

In some cases the yaml uses a list instead of a map.


Related issues 3 (2 open1 closed)

Related to Suricata - Task #2309: SuriCon 2017 brainstormAssignedVictor JulienActions
Related to Suricata - Bug #3143: datasets: don't use list in global configClosedVictor JulienActions
Related to Suricata - Feature #3636: eve: configuration options to enable all, none or just a default set of outputsAssignedOISF DevActions
Actions #1

Updated by Victor Julien almost 7 years ago

  • Related to Task #2309: SuriCon 2017 brainstorm added
Actions #2

Updated by Andreas Herz over 5 years ago

  • Tracker changed from Bug to Optimization
Actions #3

Updated by Victor Julien about 5 years ago

  • Related to Bug #3143: datasets: don't use list in global config added
Actions #4

Updated by Philippe Antoine almost 2 years ago

Make outputs.1.eve-log.types.6.files.force-hash=sha256 simpler for the user

Actions #5

Updated by Jason Ish almost 2 years ago

Philippe Antoine wrote in #note-4:

Make outputs.1.eve-log.types.6.files.force-hash=sha256 simpler for the user

Probably makes sense for the types under eve-log to be a simple map. As the following doesn't really make sense:

outputs:
  - eve-log:
      types:
        - alert
        - alert
        - anomaly

so it makes more sense like:
outputs:
  - eve-log:
      types:
        alert:
        anomaly:

Note: Its a little odd when not a list as you have to create empty objects for entries with no options.

This may also conflict with some ideas here: https://redmine.openinfosecfoundation.org/issues/3636

This would change the override notation to outputs.1.eve-log.types.files.force-hash=sha256 which makes a lot more sense.

Further simplication could be made by removing the one object object by modifying the YAML to be like:

outputs:
  - type: fast
    enabled: yes
    filename: fast.log
  - type: eve-log
    enabled: yes
    filename: eve.json
    types:
      alert:
      anomaly:

Further reducing the notation to outputs.1.types.files.force-hash=sha256.

Actions #6

Updated by Jason Ish almost 2 years ago

  • Related to Feature #3636: eve: configuration options to enable all, none or just a default set of outputs added
Actions #7

Updated by Victor Julien almost 2 years ago

Would it make sense to do a variant of

outputs:
  eve:
    filename: eve.json
    ...

    additional:
    - filename: eve1.json
      ...
    - filename: eve2.json

This would have a clean and simple default case, while supporting the flexible additional instances as well.

Actions #8

Updated by Jason Ish almost 2 years ago

Jason Ish wrote in #note-5:

Philippe Antoine wrote in #note-4:

Make outputs.1.eve-log.types.6.files.force-hash=sha256 simpler for the user

Probably makes sense for the types under eve-log to be a simple map. As the following doesn't really make sense:
[...]
so it makes more sense like:
[...]

Note: Its a little odd when not a list as you have to create empty objects for entries with no options.

This may also conflict with some ideas here: https://redmine.openinfosecfoundation.org/issues/3636

This would change the override notation to outputs.1.eve-log.types.files.force-hash=sha256 which makes a lot more sense.

Further simplication could be made by removing the one object object by modifying the YAML to be like:
[...]

Further reducing the notation to outputs.1.types.files.force-hash=sha256.

I should note that specifying the "type" of output is actually an idea from Filebeat: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html

Actions #9

Updated by Jason Ish almost 2 years ago

Victor Julien wrote in #note-7:

Would it make sense to do a variant of
[...]
This would have a clean and simple default case, while supporting the flexible additional instances as well.

Not a bad idea. I think I'd simplify a bit by allowing both of the following:

outputs:
  eve:
   - filename: just-dns.json
     types:
      dns:
   - filename: just-alerts.json
     types:
       alert:
outputs:
  eve:
    filename: eve.json
    types:
      alert:
      dns:

Testing a runtime if eve is a list or a map is rather trivial. And clears up confusion, at least for me as to what configuration applies to the additional, what does it inherit from the upper object, etc.

Does it make sense for any of the other outputs to be specified for that once?

Actions

Also available in: Atom PDF