Suricata

I confirm, this would be great. It's a problem I'm frequently having trouble with. For now I'm just checking whether flow_id value has been seen before or not, but it's a rather complex and costly way of doing it. It's also causing troubles when I'm looking to graph data coming from eve.json, there's no easy way to know if the connexion is related to a former event or not.

This feature may be related: https://redmine.openinfosecfoundation.org/issues/2628 and I think both could be merged.

Some work has been done by Stian, but I think we need a bit more on the QA side to validate that the field is always correct, see: https://github.com/OISF/suricata/pull/3521#issuecomment-446604186

Help on this would be much appreciated!

Hi Victor,
I'm going to have a serious look at it and will report on my tests.
Thanks a lot for the update.

Hi Victor,
Got the work done by Stian patched on both 3.2 and master branch in production, as well as on brand new installs over the last day.
With libhtp installed and a ./configure --enable-hiredis --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/.
The field is empty only in cas of app_proto failed (flow timeout) or when tcp_state is closed, which seems OK to me.
I tested the master branch with ../suricata-verify-master/run.py > results.log, here is the result:
@
===> alert-testmyids: OK
===> alert-testmyids-not-established: OK
===> dnp3: OK
===> dnp3-dnp3_data-alert: OK
===> dnp3-dnp3_func-alert: OK
===> dns-eve: OK
===> dns-eve-v2-udp-dig-a-www-suricata-ids-org: OK
===> dns-json-log: OK
===> dns-lua-rules: SKIPPED: requires feature HAVE_LUA
===> dns-single-request: OK
===> dns-tcp-multirequest-buffer-1: OK
===> dns-tcp-ts-gap: OK
===> dns-tcp-www-google-com: OK
===> dns-udp-dig-a-www-suricata-ids-org: OK
===> dns-udp-dns-log-unanswered: OK
===> dns-udp-double-request-response: OK
===> dns-udp-eve-log-aaaa-only: OK
===> dns-udp-eve-log-answer-only: OK
===> dns-udp-eve-log-mx-only: OK
===> dns-udp-eve-log-query-only: OK
===> dns-udp-eve-log-txt: OK
===> dns-udp-nxdomain-soa: OK
===> dns-udp-unsolicited-response: OK
===> dns-udp-z-flag-fp: OK
===> eve-alert-metadata-defaults: OK
===> eve-alert-metadata-enable-rule: OK
===> eve-alert-metadata-off: OK
===> eve-metadata: OK
===> filestore-v2.1-forced: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.2-forced-with-open-files: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.3-fserror: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.4-forced-with-meta: SKIPPED: requires feature HAVE_NSS
===> filestore-v2.5-both-enabled: SKIPPED: requires feature HAVE_NSS
===> http-xff-eve-forward-extra-data: OK
===> http-xff-eve-forward-overwrite: OK
===> http-xff-eve-reverse-extra-data: OK
===> http-xff-eve-reverse-overwrite: OK
===> http-xff-unified2: SKIPPED: requires script returned false
===> linktype-228: OK
===> lua-output-dns: SKIPPED: requires feature HAVE_LUA
===> lua-output-http: SKIPPED: requires feature HAVE_LUA
===> lua-output-smtp: SKIPPED: requires feature HAVE_LUA
===> output-eve-fileinfo: OK
===> output-pcap-log: OK
===> output-tcp-data: OK
===> proto-mismatch-http-ssh: OK
===> show-help: OK
===> smtp: OK
===> test-config-empty-rule-file: OK
===> tls: OK
===> tls-fingerprint-alert: OK
===> tls-json-output-ids: OK
===> tls-json-output-ips: OK

PASSED: 43
FAILED: 0
SKIPPED: 10
@
So this look fine to me, the feature gives great enhancement for my SIEM, especially for graphs and live maps, as well as for operators quick understanding of flows.
Please let me know if you need more testing.

Feature request #2628 can be closed (fixed by this feature request).

Hi Daniel, what I meant was a new set of tests for suricata-verify to show that the new functionality works correctly in the various scenarios. Unless I'm missing something I don't see that in the output above, correct?