Task #2167
opentracking: eve enhancements
Description
Parts of EVE are not working as well as we want. Changing those would be a breaking change in some cases. Consider a new version of eve that is free to break things.
Some initial notes:
- config format: the 'types' list should be a map instead
- config format: much too verbose
- config format 'eve-log' should just be 'eve'
- config should have better defaults
- output of buffers, see #2166
- output of DNS is too verbose #2086 #1198
- output HTTP: fix typos like #2000
- output HTTP: http.http_user_agent is redundant. Could just be http.user_agent
Updated by Andreas Herz over 7 years ago
- Assignee set to OISF Dev
- Target version set to TBD
Would the old one be deprecated someday in the future or will they coexist for a longtime?
Updated by Victor Julien over 7 years ago
Goal here is first to see what we would like to modify. Then we can see if it can be done within the current eve or that we need a fork. If the latter we'll have to consider how that would work out. But for now I would like to focus on what kind of changes we'd like to make to the input.
Updated by Jason Ish about 7 years ago
I believe we had a discussion on eve versioning, but it must have been online as I can't find any record of it.
On reviewing https://github.com/OISF/suricata/pull/2884, we see https://github.com/OISF/suricata/pull/2884/files#diff-cd6d639bc75c8f4e5201806504a446beR281 which ups the version of "eve" to 2, but only for a DNS change. If we follow this, we'll have to up it to 3 when we make a change to something like TLS for example.
Instead we maybe need 2 levels of version. An eve version for the outer layer of the "base" eve object, then a version for the event type. So using the above PR as an example, only DNS would be upped to version 2. The "dns" object would contain a "version" field of 2.
The configuration file would set the version. The default configuration file always containing the latest version. So on upgrade, if the user does not upgrade their configuration file they stick with the old version - perhaps with deprecation warnings. While I don't expect version updates often, we will want to limit the number of versions being maintained.
This does NOT address the issue of breaking eve log readers that may expect certain fields, or certain fields to be of a certain type that Victor has brought up. But that is not really related to versioning.
Updated by Victor Julien about 7 years ago
Good points Jason, I agree. Just adding here that it requires us to carefully document each version and subversion.
Updated by Jason Ish about 7 years ago
Something else that we might consider is how we name the "list" items under the outputs. Right now we have:
outputs: - fast: enabled: yes - eve-log: enabled: yes
This adds one level of extra hierarchy than is probably needed. And means the following is actually valid YAML:
outputs: - fast: enabled: yes eve-log: enabled: yes
What might be better is where the log type is set as a type field:
outputs: - type: fast enabled: yes - type: eve-log enabled: yes
This one is a bit subjective, but I think the current way adds one more level of complexity and indirection.
Updated by Victor Julien over 5 years ago
- Related to Bug #2000: eve.http: http_refer should be http_referer added
Updated by Victor Julien over 1 year ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Jason Ish
Updated by Philippe Antoine about 1 year ago
- Related to Task #6443: Suricon 2023 brainstorm added
Updated by Philippe Antoine about 1 year ago
Question : do we keep multiple possible (configurable) outputs in the same suricata version cf http.http_user_agent
vs http.user_agent
Sascha says that having both may make suricata major version bumping easier for some users
Updated by Victor Julien about 1 year ago
- Related to Bug #6458: eve/http: discrepancy in http events and http objects logged in alerts added
Updated by Victor Julien 10 months ago
- Related to Bug #6400: log of DNS answer is in wrong direction added
Updated by Victor Julien 10 months ago
- Related to Feature #4853: eve: Add information about Suricata version added
Updated by Jason Ish 5 months ago
- Related to Feature #7101: eve: add number of flowbits in protocol records and alerts added