Project

General

Profile

Actions

Feature #2167

open

eve-ng

Added by Victor Julien over 4 years ago. Updated almost 4 years ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

Parts of EVE are not working as well as we want. Changing those would be a breaking change in some cases. Consider a new version of eve that is free to break things.

Some initial notes:

  • config format: the 'types' list should be a map instead
  • config format: much too verbose
  • config format 'eve-log' should just be 'eve'
  • config should have better defaults
  • output of buffers, see #2166
  • output of DNS is too verbose #2086 #1198
  • output HTTP: fix typos like #2000
  • output HTTP: http.http_user_agent is redundant. Could just be http.user_agent

Related issues

Related to Bug #2000: eve.http: http_refer should be http_refererAssignedStian BergsethActions
Actions #1

Updated by Victor Julien over 4 years ago

  • Description updated (diff)
Actions #2

Updated by Andreas Herz about 4 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD

Would the old one be deprecated someday in the future or will they coexist for a longtime?

Actions #3

Updated by Victor Julien about 4 years ago

Goal here is first to see what we would like to modify. Then we can see if it can be done within the current eve or that we need a fork. If the latter we'll have to consider how that would work out. But for now I would like to focus on what kind of changes we'd like to make to the input.

Actions #4

Updated by Jason Ish almost 4 years ago

I believe we had a discussion on eve versioning, but it must have been online as I can't find any record of it.

On reviewing https://github.com/OISF/suricata/pull/2884, we see https://github.com/OISF/suricata/pull/2884/files#diff-cd6d639bc75c8f4e5201806504a446beR281 which ups the version of "eve" to 2, but only for a DNS change. If we follow this, we'll have to up it to 3 when we make a change to something like TLS for example.

Instead we maybe need 2 levels of version. An eve version for the outer layer of the "base" eve object, then a version for the event type. So using the above PR as an example, only DNS would be upped to version 2. The "dns" object would contain a "version" field of 2.

The configuration file would set the version. The default configuration file always containing the latest version. So on upgrade, if the user does not upgrade their configuration file they stick with the old version - perhaps with deprecation warnings. While I don't expect version updates often, we will want to limit the number of versions being maintained.

This does NOT address the issue of breaking eve log readers that may expect certain fields, or certain fields to be of a certain type that Victor has brought up. But that is not really related to versioning.

Actions #5

Updated by Victor Julien almost 4 years ago

Good points Jason, I agree. Just adding here that it requires us to carefully document each version and subversion.

Actions #6

Updated by Jason Ish almost 4 years ago

Something else that we might consider is how we name the "list" items under the outputs. Right now we have:

outputs:
  - fast:
      enabled: yes
  - eve-log:
      enabled: yes

This adds one level of extra hierarchy than is probably needed. And means the following is actually valid YAML:
outputs:
  - fast:
      enabled: yes
    eve-log:
      enabled: yes

What might be better is where the log type is set as a type field:

outputs:
  - type: fast
    enabled: yes
  - type: eve-log
    enabled: yes

This one is a bit subjective, but I think the current way adds one more level of complexity and indirection.

Actions #7

Updated by Victor Julien about 2 years ago

  • Related to Bug #2000: eve.http: http_refer should be http_referer added
Actions

Also available in: Atom PDF