Project

General

Profile

Actions

Bug #1938

closed

suricata: log handling issues

Added by Arturo Borrero Gonzalez over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Hi,

the log files handling in suricata probably requires a bit of tunning.

In debian, we have two issues with this: * log files rotation (when it happens due to logrotate) * log files reopen (when that happens due to suricata reload)

The second option is specially interesting when running suricata with another user than root.

Two debian related bugs: * https://bugs.debian.org/842049 (suricata loses log output after log rotation) * https://bugs.debian.org/836929 (suricata should drop root privileges when running)

I decided to merge both in a single request here for clarity, given both are related to the same.

Actions #1

Updated by Andreas Herz over 7 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD

I think they are related but two different issues. Especially for the second case we have similiar issues like https://redmine.openinfosecfoundation.org/issues/1837 which seem to have some Ubuntu/Debian specific. On CentOS based systems I have it running with a dedicated suricata user that also has dedicated permissions for the /var/log/suricata folder and /etc/suricata config folder.

So the second issue needs more details.

With the first issue is the question if it also happens with a simple -HUP signal.

Could you check if the second issue might fit in the one from 1837 so we "merge" them?
And could you try to produce a simple case to reproduce it to narrow it down?

Thanks!

Actions #2

Updated by Arturo Borrero Gonzalez over 7 years ago

Here the steps to reproduce the log rotation issue:

1) logrotate configuration at /etc/logrotate.d/suricata


/var/log/suricata/*.log
/var/log/suricata/*.json
{
        rotate 14
        missingok
        compress
        create
        sharedscripts
        postrotate
                /bin/kill -HUP $(cat /var/run/suricata.pid)
        endscript
}

2) Start suricata:

systemctl start suricata

3) check that there are logs:

tail /var/log/suricata/suricata.log

4) check that you can produce more logs:

suricatasc -c reload-rules && tail /var/log/suricata/suricata.log

5) force log rotation (will kill -HUP suricata)

logrotate --force /etc/logrotate.d/suricata

6) no further log entries in the log:

suricatasc -c reload-rules && tail /var/log/suricata/suricata.log

Actions #3

Updated by Arturo Borrero Gonzalez over 7 years ago

Regarding the issues #1837 with dropping permissions, yes, I think they are different issues.

Perhaps it just a matter of ordering the logfile creation/opening and the privilege dropping at suricata startup time.

Actions #4

Updated by Andreas Herz over 7 years ago

I played with that on a Debian system with 3.1.3 stable installed and configured with defaults and as root.
So far I see no issue, could you try this with your setup as well?

  1. /usr/local/bin/suricata -c /usr/local/etc/suricata/suricata.yaml -i eth0 -vv
  2. tail -f /usr/local/var/log/suricata/suricata.log
  3. kill -USR2 $(pidof suricata)
  4. kill -HUP $(pidof suricata)
  5. kill -USR2 $(pidof suricata)

(tail and kill on dedicated tmux screens)

I still see the suricata.log being updated. I don't think it's a "logrotate" issue but more with the HUP signal in your scenario.
If my steps already fail for you we have a more easy setup to debug :)

Actions #5

Updated by Sascha Steinbiss over 7 years ago

Hi Andreas,

here's some input from me as well, as I'm one of the people who noticed this issue in production.

In the logrotate scenario I was referring to, suricata.log is renamed and recreated (logrotate option 'create') while suricata is running before sending SIGHUP.
It looks like the rotation_flag set by SIGHUP is only evaluated by SCLogFileWrite() -- as used for writing eve.json and friends -- not by SCLogMessage() -- as used for writing suricata.log -- the new file is not reopened.

Hence, from what I can see, as long as the suricata.log stays in place, you won't see this issue surfacing.
My first idea was to simply use 'copytruncate' instead of 'create' in the logrotate config, which will preserve the output file descriptor, but as SCLogInitFileOPIface() does not open suricata.log in append mode, the result is still not as expected.

IMHO one would need to also add rotation flag handling in SCLogMessage() and properly reopen files that have been moved away. If I am missing something I'd be happy to learn :)

Sascha

Actions #6

Updated by Jason Ish over 7 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Jason Ish
Actions #7

Updated by Jason Ish over 7 years ago

  • Status changed from Assigned to Closed
  • Target version changed from TBD to 3.2
Actions

Also available in: Atom PDF