Feature #2565
closedRegular and verbose status output should go to STDOUT, not STDERR
Description
When outputting information to the screen, regular output should only go to STDOUT, not STDERR. STDERR should only be used for errors. Currently all output goes to STDERR. This makes it impossible to separate what is normal output and what are errors.
It is important to separate the streams for a tool like this that is run through cron. Cron will email output of a command (both STDOUT and STDERR) to the admin, but this is often not desired during normal operation. The typical cron setup is to redirect STDOUT to a log file, and allow STDERR to output normally. This way, if there is an error, cron will send an email with the error message, and does not create noise otherwise during normal operation.
As-is, it is necessary to create fragile pipelines and other methods to try to determine if there was an error during the update process. Application authors are much more suited to determine what is an error, and should use the correct output stream based on that knowledge.
Updated by Jason Ish over 6 years ago
So the way things are done in suricata-update is that anything with the timestamp at the beginning is a log line (diagnostic output). All logs go to stderr. This isn't much different than many other apps.
Interactive output, like that from "suricata-update list-sources" does go to stdout.
I guess I can see how this can be an issue with cron if you want errors to generate an email, but have a report output to a file. I think this might require more thought than simply sending info to stdout and warning/error to stderr.
In the cron case, if a runtime report is not desired, "-q" can be used which will only output warning/errors.
Updated by B Mathis over 6 years ago
Jason Ish wrote:
This isn't much different than many other apps.
Two wrongs don't make a right. I do see other apps that output to stderr, and I open bug reports for them too. The usage is pretty clear and codified in the name, stdout for regular output, and stderr for errors (not necessarily warnings - only things that prevent the app from performing it's function and need manual intervention to resolve).
I guess I can see how this can be an issue with cron if you want errors to generate an email, but have a report output to a file. I think this might require more thought than simply sending info to stdout and warning/error to stderr.
This is pretty much the canonical use of cron, which would seem to be the main way suricata-update is meant to be run. The current approach is possibly a result of treating stdout and stderr as generic output streams as opposed to the traditional and well-accepted use of them. Interaction and regular output takes place on stdout on all standard apps.
In the cron case, if a runtime report is not desired, "-q" can be used which will only output warning/errors.
But a report is desired, and should be for most uses of the app, so that's not an escape route or workaround here.
There can be cases where this convention is not followed, but I think they are very rare and require evaluation based on the expected typical usage of the application. In the case of suricata-update, I would say the overwhelming use case is that it's run from cron, and interactive use is virtually non-existent except for when you initially set it up. If someone chooses to redirect the output to something else, they know what they're doing and can deal with the side-effects.
Cron is dumb and shouldn't need to know the specific cases of every app, and admins at this stage of the process shouldn't need to know either. The app needs to tell cron what's going on in a way it has always understood things, which is via output streams. Only when the info makes it into the log should the tags become relevant for the person reading them, and they can separate info, debug, etc.
I will amend my request to say that errors should go to BOTH stdout and stderr, so they generate alerts and are also recorded in logs.
Updated by B Mathis over 6 years ago
To make sure this isn't lost in the text of my last update:
I will amend my request to say that errors should go to BOTH stdout and stderr, so they generate alerts and are also recorded in logs.
Updated by Shivani Bhardwaj over 5 years ago
- Status changed from New to Assigned
- Assignee changed from Jason Ish to Vagisha Gupta
- Target version set to TBD
Updated by Jason Ish about 5 years ago
- Status changed from Assigned to Closed
- Target version changed from TBD to 1.1.0rc1
A fix has been applied to master and will be in the next release.