Project

General

Profile

Actions

Feature #2644

open

Add direction of stream to eve-json events

Added by Stian Bergseth almost 3 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Add a direction field in eve-json to indicate what direction the stream started with.


Related issues

Related to Feature #2628: Specify the flow direction in metadata sent by Suricata.AssignedDaniel JaraudActions
Actions #1

Updated by Daniel Jaraud almost 3 years ago

I confirm, this would be great. It's a problem I'm frequently having trouble with. For now I'm just checking whether flow_id value has been seen before or not, but it's a rather complex and costly way of doing it. It's also causing troubles when I'm looking to graph data coming from eve.json, there's no easy way to know if the connexion is related to a former event or not.

Actions #2

Updated by Daniel Jaraud almost 3 years ago

This feature may be related: https://redmine.openinfosecfoundation.org/issues/2628 and I think both could be merged.

Actions #3

Updated by Victor Julien almost 3 years ago

Some work has been done by Stian, but I think we need a bit more on the QA side to validate that the field is always correct, see: https://github.com/OISF/suricata/pull/3521#issuecomment-446604186

Help on this would be much appreciated!

Actions #4

Updated by Daniel Jaraud almost 3 years ago

Hi Victor,
I'm going to have a serious look at it and will report on my tests.
Thanks a lot for the update.

Actions #5

Updated by Daniel Jaraud almost 3 years ago

Hi Victor,
Got the work done by Stian patched on both 3.2 and master branch in production, as well as on brand new installs over the last day.
With libhtp installed and a ./configure --enable-hiredis --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/.
The field is empty only in cas of app_proto failed (flow timeout) or when tcp_state is closed, which seems OK to me.
I tested the master branch with ../suricata-verify-master/run.py > results.log, here is the result:
@
===> alert-testmyids: OK
===> alert-testmyids-not-established: OK
===> dnp3: OK
===> dnp3-dnp3_data-alert: OK
===> dnp3-dnp3_func-alert: OK
===> dns-eve: OK
===> dns-eve-v2-udp-dig-a-www-suricata-ids-org: OK
===> dns-json-log: OK
===> dns-lua-rules: SKIPPED: requires feature HAVE_LUA
===> dns-single-request: OK
===> dns-tcp-multirequest-buffer-1: OK
===> dns-tcp-ts-gap: OK
===> dns-tcp-www-google-com: OK
===> dns-udp-dig-a-www-suricata-ids-org: OK
===> dns-udp-dns-log-unanswered: OK
===> dns-udp-double-request-response: OK
===> dns-udp-eve-log-aaaa-only: OK
===> dns-udp-eve-log-answer-only: OK
===> dns-udp-eve-log-mx-only: OK
===> dns-udp-eve-log-query-only: OK
===> dns-udp-eve-log-txt: OK
===> dns-udp-nxdomain-soa: OK
===> dns-udp-unsolicited-response: OK
===> dns-udp-z-flag-fp: OK
===> eve-alert-metadata-defaults: OK
===> eve-alert-metadata-enable-rule: OK
===> eve-alert-metadata-off: OK
===> eve-metadata: OK
===> filestore-v2.1-forced: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.2-forced-with-open-files: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.3-fserror: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.4-forced-with-meta: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.5-both-enabled: SKIPPED: requires feature HAVE_NSS
===> http-xff-eve-forward-extra-data: OK
===> http-xff-eve-forward-overwrite: OK
===> http-xff-eve-reverse-extra-data: OK
===> http-xff-eve-reverse-overwrite: OK
===> http-xff-unified2: SKIPPED: requires script returned false
===> linktype-228: OK
===> lua-output-dns: SKIPPED: requires feature HAVE_LUA
===> lua-output-http: SKIPPED: requires feature HAVE_LUA
===> lua-output-smtp: SKIPPED: requires feature HAVE_LUA
===> output-eve-fileinfo: OK
===> output-pcap-log: OK
===> output-tcp-data: OK
===> proto-mismatch-http-ssh: OK
===> show-help: OK
===> smtp: OK
===> test-config-empty-rule-file: OK
===> tls: OK
===> tls-fingerprint-alert: OK
===> tls-json-output-ids: OK
===> tls-json-output-ips: OK

PASSED: 43
FAILED: 0
SKIPPED: 10
@
So this look fine to me, the feature gives great enhancement for my SIEM, especially for graphs and live maps, as well as for operators quick understanding of flows.
Please let me know if you need more testing.

Actions #6

Updated by Daniel Jaraud almost 3 years ago

Feature request #2628 can be closed (fixed by this feature request).

Actions #7

Updated by Victor Julien almost 3 years ago

Hi Daniel, what I meant was a new set of tests for suricata-verify to show that the new functionality works correctly in the various scenarios. Unless I'm missing something I don't see that in the output above, correct?

Actions #8

Updated by Andreas Herz over 2 years ago

  • Target version set to TBD
Actions #9

Updated by Victor Julien about 2 years ago

  • Related to Feature #2628: Specify the flow direction in metadata sent by Suricata. added
Actions

Also available in: Atom PDF