Project

General

Profile

Actions

Feature #1454

closed

Proposal to add Lumberjack/CEE formatting option to EVE JSON syslog output for compatibility with rsyslog parsing

Added by Zach Rasmor almost 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

I would like to propose a feature enhancement to make the eve-log syslog output format compatible with the Lumberjack/CEE format required by the mmjsonparse modification module of rsyslog.

The rsyslog mmjsonparse module requires the JSON portion of the message to be prepended by the “CEE cookie”, the character sequence “@cee:” (http://www.rsyslog.com/doc/mmjsonparse.html)
The mmjsonparse module parses JSON messages into their key-value pairs, allowing for direct access to relevant message fields. This is an alternative approach to relying on “contains” to essentially perform a string search on the JSON string. This is especially helpful when accessing the ‘event_type’ field for the purpose of redirecting JSON output from various event types to different files/locations.

I created the following patch to test this and the results have been positive. I propose adding a line in the eve-log.syslog section of the yaml file (line range 99-101) that is called “lumberjack-cee-format: yes/no”, which will be evaluated in an if/else block that will enclose the patched area below.

--- suricata-2.1beta3/src/output-json.c 2015-01-29 15:52:53.000000000 0000
++ suricata-2.1beta3/src/output-json-lumberjack.c 2015-04-20 15:46:29.973693576 +0000
@ -346,7 +346,7 @

SCMutexLock(&file_ctx->fp_mutex);
if (json_out ALERT_SYSLOG) {
- syslog(alert_syslog_level, "%s", js_s);
+ syslog(alert_syslog_level, "%s%s", "@cee: ", js_s);
} else if (json_out ALERT_FILE || json_out ALERT_UNIX_DGRAM || json_out ALERT_UNIX_STREAM) {
MemBufferWriteString(buffer, "%s\n", js_s);
file_ctx->Write((const char *)MEMBUFFER_BUFFER(buffer),

I am willing to perform the development for this enhancement if there are no objections or additional feedback.


Related issues 1 (0 open1 closed)

Related to Suricata - Bug #1602: eve-log prefix field feature brokenClosedEric Leblond11/25/2015Actions
Actions #1

Updated by Eric Leblond almost 9 years ago

Very good idea, I was also considering to do the same. The "lumberjack-cee-format: yes/no" is a bit too long so I would propose "cee-format: yes/no" instead. But you will ba able to add a comment to the yaml to specify that it is useful for both things.

Actions #2

Updated by Peter Manev almost 9 years ago

I second that

Actions #3

Updated by Zach Rasmor almost 9 years ago

Thanks for the suggestion, I agree that 'cee_format' is cleaner. I have finished development for this feature, and should be finished testing today. In the meantime, can somebody tell me how I can get the "Developer" role so that I can claim this ticket? Thanks!

Actions #4

Updated by Victor Julien almost 9 years ago

  • Status changed from New to Assigned
  • Assignee set to Zach Rasmor
  • Target version set to 2.1beta4
Actions #5

Updated by Zach Rasmor almost 9 years ago

  • % Done changed from 0 to 100

Pull Request #1449 submitted. (Also depends on fix for Bug #1204, for which I submitted Pull Request #1447)

Actions #6

Updated by Victor Julien almost 9 years ago

  • Target version changed from 2.1beta4 to 3.0RC1
Actions #7

Updated by Victor Julien over 8 years ago

  • Status changed from Assigned to Closed
Actions #8

Updated by Zach Rasmor over 8 years ago

  • Related to Bug #1602: eve-log prefix field feature broken added
Actions

Also available in: Atom PDF