Suricata

Goal here is first to see what we would like to modify. Then we can see if it can be done within the current eve or that we need a fork. If the latter we'll have to consider how that would work out. But for now I would like to focus on what kind of changes we'd like to make to the input.

I believe we had a discussion on eve versioning, but it must have been online as I can't find any record of it.

On reviewing https://github.com/OISF/suricata/pull/2884, we see https://github.com/OISF/suricata/pull/2884/files#diff-cd6d639bc75c8f4e5201806504a446beR281 which ups the version of "eve" to 2, but only for a DNS change. If we follow this, we'll have to up it to 3 when we make a change to something like TLS for example.

Instead we maybe need 2 levels of version. An eve version for the outer layer of the "base" eve object, then a version for the event type. So using the above PR as an example, only DNS would be upped to version 2. The "dns" object would contain a "version" field of 2.

The configuration file would set the version. The default configuration file always containing the latest version. So on upgrade, if the user does not upgrade their configuration file they stick with the old version - perhaps with deprecation warnings. While I don't expect version updates often, we will want to limit the number of versions being maintained.

This does NOT address the issue of breaking eve log readers that may expect certain fields, or certain fields to be of a certain type that Victor has brought up. But that is not really related to versioning.

Good points Jason, I agree. Just adding here that it requires us to carefully document each version and subversion.

Something else that we might consider is how we name the "list" items under the outputs. Right now we have:

outputs:
  - fast:
      enabled: yes
  - eve-log:
      enabled: yes

outputs:
  - fast:
      enabled: yes
    eve-log:
      enabled: yes

What might be better is where the log type is set as a type field:

outputs:
  - type: fast
    enabled: yes
  - type: eve-log
    enabled: yes

This one is a bit subjective, but I think the current way adds one more level of complexity and indirection.

Question : do we keep multiple possible (configurable) outputs in the same suricata version cf http.http_user_agent vs http.user_agent
Sascha says that having both may make suricata major version bumping easier for some users