Suricata

Currently multiple include statements can be provided in the Suricata configuration like:

include: somefile.yaml
include: some-other-file.yaml

However, this is invalid YAML as duplicate keys are forbidden, and some YAML parsers, in particular Rust serde_yaml will error out on these duplicate keys. Other parsers may as well, or only keep one of these values.

This works for us as we use a rather low level event emitting YAML parser and can provide our own "magic" to the YAML. It would be nice to move to a YAML format that is parseable by any third party parser.

Multiple include files could still be used at the same level by using an array:

include:
  - somefile.yaml
  - some-other-file.yaml

As order shouldn't change the resulting YAML, setups like the following could be converted:

include: somefile.yaml

outputs:
 - ...

include: some-other-file.yaml

Also, include statements at different levels would still be supported (however, this has never been officially tested, but appears to work)

include: outputs.yaml
vars:
  include: vars.yaml

With Serde, or any other higher level parser we'd be presented with a rather abstract tree of values that we'd would then parse and resolve the includes, so it would be a 2 stage parser.

The main benefit here is to move away from essentially is our own YAML parsing implementation allowing us to use libraries to completely parse the YAML.

I'd like to deprecate with a warning multiple include statements for 7.0 so we can transition YAML libraries for 8.0.