Project

General

Profile

Actions
Copy link

Documentation #2699

closed

document all eve record types and fields

Added by Victor Julien over 6 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Sascha Steinbiss
Target version:
TBD
Affected Versions:
Effort:
medium
Difficulty:
Label:

Description

For each document type, document fields and their types. Add examples.

It's probably best to add specific tickets for each of the record types.

Files

Download all files
sphinx1.png (124 KB) sphinx1.png Sascha Steinbiss, 11/09/2023 11:26 AM
sphinx2.png (117 KB) sphinx2.png Sascha Steinbiss, 11/09/2023 11:26 AM

Related issues 4 (4 open — 0 closed)

Related to Suricata - Task #2685: SuriCon 2018 brainstormAssignedVictor JulienActions
Related to Suricata - Documentation #2620: userguide: document tagged_packets / event_type packetNewOISF DevActions
Related to Suricata - Documentation #5359: userguide: improve documentation on (main) EVE fieldsNewOISF DevActions
Related to Suricata - Documentation #6071: eve/schema: add descriptions to the schemaAssignedJason IshActions
Actions
Copy link
 #1

Updated by Victor Julien over 6 years ago

  • Assignee set to Robert Haist
  • Target version set to TBD
Actions
Copy link
 #2

Updated by Victor Julien over 6 years ago

  • Related to Task #2685: SuriCon 2018 brainstorm added
Actions
Copy link
 #3

Updated by Victor Julien over 6 years ago

Actions
Copy link
 #4

Updated by Robert Haist over 6 years ago

  • Effort set to medium

Working on it over at Github: https://github.com/rhaist/suricata-json-schema

Will probably take some time until we have a fully reproducible build but you can expect a preview soon.

Actions
Copy link
 #5

Updated by Victor Julien over 6 years ago

Maybe I'm misunderstanding the purpose of the schema, but the goal of this ticket is to get the userguide updated so that all missing EVE fields are documented.

The JSON schema ticket is #1369

Actions
Copy link
 #6

Updated by Jason Ish about 6 years ago

Victor Julien wrote:

Maybe I'm misunderstanding the purpose of the schema, but the goal of this ticket is to get the userguide updated so that all missing EVE fields are documented.

The JSON schema ticket is #1369

I think the 2 are tightly related.

I had started to look at this again, more about how it should look to the end user. I played with using tables in Sphinx, but I don't find that scales well, especially if you want to reformat. When I jumped back to my JSON schema stuff, it is kind of ugly and I'm not sure if it can be used to generate suitable doc for the userguide. So my last attempt is just some custom YAML that I thought I might generate into Sphinx tables. Still not sure if that is a good idea though, given that JSON schema exists.

Ideally there should be one source of truth. If we still feel that JSON schema is suitable for QA testing, maybe that should be it. We could probably do some intermediate processing of it, and perhaps adding extra fields to provide context in end-user doc. By context I mean stuff like: "vlan - only present when the alerting packet has a vlan header".

Actions
Copy link
 #7

Updated by Andreas Herz almost 6 years ago

  • Assignee changed from Robert Haist to Sascha Steinbiss
Actions
Copy link
 #8

Updated by Andreas Herz almost 6 years ago

  • Tracker changed from Feature to Documentation
Actions
Copy link
 #9

Updated by Philippe Antoine over 2 years ago

@Sascha Steinbiss is this done through the json schema ?

Actions
Copy link
 #10

Updated by Sascha Steinbiss over 2 years ago

Good question. The generic documentation (as in: RTD pages) is not there yet, but from my point of view the JSON schema is OK. (Is etc/schema.json based on our 2019 stuff BTW or separately generated?)

Actions
Copy link
 #11

Updated by Philippe Antoine over 2 years ago

Good question. The generic documentation (as in: RTD pages) is not there yet, but from my point of view the JSON schema is OK. (Is etc/schema.json based on our 2019 stuff BTW or separately generated?)

I do not know your 2019 stuff.
etc/schema.json was generated with combining the outputs of suricata-verify tests (and then manual additions over time)

Actions
Copy link
 #12

Updated by Philippe Antoine over 2 years ago

Actions
Copy link
 #13

Updated by Jason Ish over 2 years ago

I've been working on a script to generate output from the schema.. Rough example here: https://gist.github.com/jasonish/fc04da8a5586954f78e1857fe3ae0203.

I'm thinking the next step would be a rather simple, but large `.rst` rendering as an appending in the docs with predictable anchors so one can link to the relevant section by protocol name, etc.

Actions
Copy link
 #14

Updated by Philippe Antoine over 1 year ago

Actions
Copy link Download all files
 #16

Updated by Sascha Steinbiss over 1 year ago

My feeling is BTW that the stuff we did for Suricon 2019 is probably obsolete:

  • There now is an official JSON schema maintained by the project itself, which is also used in tests to ensure it stays up-to-date. Much better than trying to establish one from the outside :)
  • Our hacky documentation generation script indeed creates a tree of Sphinx documentation for RTD (see attachments), but it's pretty closely tied to our project and its result tree structure. Also, I'm not sure about predictable anchors; our script creates individual pages (https://github.com/satta/suricata-json-schema/blob/master/schematize.py).

Do we even want to keep this open?

Actions
Copy link
 #17

Updated by Sascha Steinbiss about 1 year ago

  • Status changed from Assigned to Closed

I am closing this now since I agree that the JSON schema that is already being maintained addresses the initial task of enumerating the fields and https://redmine.openinfosecfoundation.org/issues/6071 already tracks the documentation.

Actions
Copy link

Also available in: Atom PDF