Feature #2168
closedRules files from Suricata sources (like decoder-events.rules) are packaged as config files and may not be updated on package update.
Description
In many environments Suricata packages are updated automatically, with tools like ansible, puppet, etc. This tools usually preserve old config files, invoking apt with options like Dpkg::Options::="--force-confold"
.
Suricata debian packages list built-in rules among conf files:
/etc/suricata/rules/files.rules /etc/suricata/rules/modbus-events.rules /etc/suricata/rules/app-layer-events.rules /etc/suricata/rules/stream-events.rules /etc/suricata/rules/smtp-events.rules /etc/suricata/rules/dnp3-events.rules /etc/suricata/rules/http-events.rules /etc/suricata/rules/decoder-events.rules /etc/suricata/rules/tls-events.rules /etc/suricata/rules/dns-events.rules
This may result in using outdated built-in rules after package update. Happened to me.
Also, regular updating of files inside /etc may cause a lot of alerts from HIDS (ossec in my case).
I suggest to move rules files ouside /etc, for example/var/lib/suricata/rules
:
- they won't be anymore considered as config files and will always be updated together with package
- HIDS will be happy that nothing changes in /etc
/etc/suricata/rules
may become symlink to /var/lib/suricata/rules
for better backward compatibility.
Updated by Andreas Herz over 7 years ago
- Assignee set to OISF Dev
- Target version set to TBD
Hmm but that's also something tools like puppet can already manage. You suggest we change the default configuration?
Updated by Pavel Selivanov over 7 years ago
I suppose, at least built-in rules should not be marked as config files in package, because they should always be updated on package update. You can't have old version of built-in rules working with new Suricata version, it produces errors. And thus they should be moved outside /etc, because everyone expects files there to be configs.
Now if you update Suricata package without overwriting configs, you have old built-in rules with new package. If you choose to update configs, you get new built-in rules, but also your suricata.yml is replaced with the default. So after update you have to use ansible/puppet/whatever to restore the correct configuration.
With almost every other package, you update package, preserving old configs, package scripts restart the service and everything is fine. It would be nice for Suricata to also be updated easily like that.
About moving all rules outside /etc - it won't break anything. And it will increase number of systems where everything works fine out-of-the-box together with HIDS. That may be controversial, but worth thinking about.
Updated by Peter Manev over 7 years ago
- Assignee changed from OISF Dev to Peter Manev
I will look into using debian conffiles to be less intrusive.
@Pavel - care to test and feedback when ready?
Updated by Pavel Selivanov over 7 years ago
@Peter_Manev - of course I'll test new packages, I am interested to have Suricata updating smoothly.
Updated by Jason Ish about 7 years ago
As there has also been some of Redmine discussion related to this topic, I thought I'd try outline a file layout that may satisfy this, plus the offline discussion.
Don't mark /etc/suricata/rules as configuration files. Let the packages/installers overwrite the files in this directory with the latest version of the rule files that are maintained in the Suricata source code. The suricata.yaml could reference these files directly, or a rule update tool could use them as input, and then they get tuned by update tool.
Alternatively, remove /etc/suricata/rules altogether and store in /usr/share/suricata/rules (actually datarootdir
) which is describe as being for "read-only architecture-independent data files". Sounds about right for rule files included with the Suricata distribution.
Introduce /var/lib/suricata (is this always localstatedir
/lib/suricata). Rules would then be stored in /var/lib/suricata/rules. These reserves all of /var/lib/suricata for future use, for example, Lua scripts that may be managed by update tools in the future.
From the LFH, "Users shouldn't ever need to modify files in /var/lib to configure a package's operation.". I think this fits the usage pattern of how most users make use of a rule update tool instead of manually managing them. At any rate, its just a convention and those that manually manage their rules can continue to do what they've always done.
Hypothetical rule update tool could now pull down rules from ET, also read in the contents of /etc/suricata/rules/*.rules, apply transforms (enable, disables, modifies, drops) and output the rules to /var/lib/suricata/rules.
This lets /etc/suricata stay static with the exception of actual configuration changes.
Permissions could also be tuned here for less operations of root. For example, make /var/lib/suricata writable by the Suricata group. Also make the control socket writable by group Suricata. Then a non-root user could update the rules and trigger a reload. But at least with permissions, that will be left largely up to the packaging.
Thoughts?
Updated by Jason Ish about 7 years ago
Something that needs to be addressed is where will users put their own custom rules. If we're looking at an end game where "suricata-update" is the defacto standard way to get new rules (like freshclam for clamav) we might look at a layout like:
/usr/share/suricata/rules/ - Rules that come with Suricata.
/etc/suricata/rules/ - User supplied custom rules, nothing is put in here by Suricata.
And then /var/lib/suricat/rules where Suricata loads the rules from with is assembled from downloaded rules (ET for example), /usr/share/suricata/rules, and /etc/suricata/rules.
Updated by Victor Julien about 7 years ago
I agree completely. So in summary with suricata-updater, we'll move to:
- sysconfdir/suricata/rules only for local rules (custom rules)
- datarootdir/suricata/rules for rules we ship (decoder and app-layer events rules)
- localstatedir/lib/suricata/rules for the output of 'suricata-updater', this is where suricata will load it's rules from
This way no real changes to Suricata are needed at all (except perhaps a new default rule path & rule list), as we load from a single source (localstatedir/lib/suricata/rules/). Changes we'd need to do:
- update make install-full to use suricata-updater (run once, display docs on installing as cronjob?)
- change suricata.yaml.in to reflect the new paths
I guess one tricky thing during this transition would be that we'd have to consider old rules files in sysconfdir/suricata/rules. Perhaps it makes sense to actually use a new directory for local rules, so that there will be no conflicts. E.g. sysconfdir/suricata/local-rules
Updated by Pavel Selivanov about 7 years ago
I like the suggested structure:
- sysconfdir/suricata/rules-local only for local rules
- datarootdir/suricata/rules for rules we ship (decoder and app-layer events rules)
- localstatedir/lib/suricata/rules for the output of 'suricata-updater', this is where suricata will load it's rules from
- sysconfigdir/suricata/rules-built-in -> datarootdir/suricata/rules
- sysconfigdir/suricata/rules-updated -> localstatedir/lib/suricata/rules
That would make the new layout more obvious for everyone.
I am not using Suricata now(left that job), but when I did I had to make some effort to make built-in rules actually update every time updating the suricata package.
Updated by Kenneth Kolano about 6 years ago
It seems like Suricata now deploys it's rules to: /usr/share/suricata/rules (at least when deploying from src). Would be good if Suricata-Update would read them from there, it still only seems to look under /etc/suricata/rules.
Updated by Jason Ish about 6 years ago
Looks like when /etc/suricata/rules exists, it will take precedence $datadir/suricata/rules. I'll create a specific issue for this case. We should look in $datadir/suricata/rules first, then fall back to other locations.
Updated by Jason Ish over 5 years ago
- Related to Bug #3138: Don't install Suricata provided rules to /etc/suricata/rules as part of make install-rules. added
Updated by Victor Julien about 5 years ago
Is there still something to do for this ticket? Seems #3138 covers this?
Updated by Pavel Selivanov about 5 years ago
Victor Julien wrote:
Is there still something to do for this ticket? Seems #3138 covers this?
Yep, I think moving rules outside /etc closes this issue.
Updated by Victor Julien about 5 years ago
- Status changed from New to Closed
- Assignee deleted (
Peter Manev) - Target version deleted (
TBD)
Thanks for the quick response Pavel!