Project

General

Profile

Actions

Feature #383

closed

Stream logging

Added by Martin Holste almost 11 years ago. Updated about 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

Create the capability to write the stream buffer to disk with a file named something like this (Vortex output):
tcp-100243088-1321557507-1321557516-r-9940-10.0.145.126:52589s66.235.132.118:80
$proto-$cnx_id-$start_ts-$end_ts-$term_reason-$bytes-$srcip:$srcport$originator$dstip:$dstport
The stream buffer should look similar to Wireshark's "follow stream" feature. Then print (or log) the name of the file so that the wrapper script doesn't have to perform expensive directory listings to find the new files being created. This could use the current configuration parameters for dictating stream cutoffs for size and time. The only new config params needed would be to specify the format, the log file to write to, and specify the directory to write streams to.

Actions #1

Updated by Victor Julien almost 11 years ago

  • Assignee set to Anonymous
  • Target version set to TBD

This would require a number of changes:

  • keeping tcp segments in a stream until the cutoff is reached
  • adding a timeout check instead of the cutoff
  • add logging output module for the index log
  • add a stream writing module for the stream files
  • create a custom runmode so it can run in a decicated mode

Assigning to the community for now.

Actions #2

Updated by Victor Julien about 8 years ago

  • Status changed from New to Closed
  • Assignee changed from Anonymous to Victor Julien
  • Target version changed from TBD to 2.1beta2
  • % Done changed from 0 to 100

This can now be done (mostly) with Lua scripts, see https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Lua_Output#Streaming-Data

Also, now have:

  # Log TCP data after stream normalization
  # 2 types: file or dir. File logs into a single logfile. Dir creates
  # 2 files per TCP session and stores the raw TCP data into them.
  # Using 'both' will enable both file and dir modes.
  #
  # Note: limited by stream.depth
  - tcp-data:
      enabled: no
      type: file
      filename: tcp-data.log

  # Log HTTP body data after normalization, dechunking and unzipping.
  # 2 types: file or dir. File logs into a single logfile. Dir creates
  # 2 files per HTTP session and stores the normalized data into them.
  # Using 'both' will enable both file and dir modes.
  #
  # Note: limited by the body limit settings
  - http-body-data:
      enabled: no
      type: file
      filename: http-data.log

Actions

Also available in: Atom PDF