Bug #7868
openChanged handling of rules in subfolders since Suricata-Update >1.3.x
Description
Hi Suricata Team,
we've noticed some inconsistent behaviour between versions while upgrading Suricata-Update from 1.2.7 (Debian Bullseye-Backports) to 1.3.4 (Debian Bookworm-Backports). We can reproduce this by the attached minimal example rules.
Given a directory structure as follows:
# tar -tf local-rules.tar.gz
rules/
rules/subfolder/
rules/rules1.rules
rules/hashes1.md5
rules/subfolder/rules2.rules
rules/subfolder/hashes2.md5
rules/subfolder/hashes3.md5
rules/subfolder/hashes4.md5
The hash-files are all identical for this demo.
The rule-files look like this:
# cat rules1.rules
alert http any any -> any any (msg:"Rule in mainfolder, hashfile in mainfolder, filename reference without subfolder"; filemd5:hashes1.md5; sid:1000001;)
alert http any any -> any any (msg:"Rule in mainfolder, hashfile in subfolder, filename reference with subfolder"; filemd5:subfolder/hashes2.md5; sid:1000002;)
# cat subfolder/rules2.rules
alert http any any -> any any (msg:"Rule in subfolder, hashfile in subfolder, filename reference without subfolder"; filemd5:hashes3.md5; sid:1000003;)
alert http any any -> any any (msg:"Rule in subfolder, hashfile in subfolder, filename reference with subfolder"; filemd5:subfolder/hashes4.md5; sid:1000004;)
In Suricata-Update 1.2.7, rule 1000003 fails ( <Error> -- filemd5 file hashes3.md5 was not found ), so we built all our rules like rule 1000004, with the 'subfolder/' in the filemd5.
Loading exactly the same local-rules.tar.gz in Suricata-Update 1.3.4, rule 1000003 is OK now, but rule 1000004 fails ( <Error> -- filemd5 file subfolder/hashes4.md5 was not found ). This is how we found this behaviour.
After some troubleshooting in the Python code, I suppose this behaviour is caused by the changed file-handling because of the work done on using hashes in the filenames. I noticed that for 1.3.4 the variable is source_filename=54f36aa263627c6935edd30702d67e76/rules/subfolder/subfolder/hashes4.md5 , having a double 'subfolder/subfolder' in it.
I can try to do a PR fixing this, but before I'd like to clarify, what the intended behaviour of Suricata-Update actually is.
Shall a filename-reference for a rule in a subfolder be relative to the rule itself, or to the final suricata.rules in the parent directory, or shall both cases be supported? (I know that in any cases the final rules have to be loaded without any errors in Suricata.)
Thanks for your answers and for your help.
Andreas
Files
Updated by Jason Ish 9 days ago
Shall a filename-reference for a rule in a subfolder be relative to the rule itself, or to the final suricata.rules in the parent directory, or shall both cases be supported? (I know that in any cases the final rules have to be loaded without any errors in Suricata.)
The reference to the filename should be relative to the file the rule is defined in. It appears in 1.2.7 this wasn't the case, but is something I would consider a bug. So I don't expect 1000004 to be handled properly and an error should be emitted.
Given that, is this still a bug?
Updated by Andreas Dolp 8 days ago
Jason Ish wrote in #note-3:
Shall a filename-reference for a rule in a subfolder be relative to the rule itself, or to the final suricata.rules in the parent directory, or shall both cases be supported? (I know that in any cases the final rules have to be loaded without any errors in Suricata.)
The reference to the filename should be relative to the file the rule is defined in. It appears in 1.2.7 this wasn't the case, but is something I would consider a bug. So I don't expect
1000004to be handled properly and an error should be emitted.Given that, is this still a bug?
Thanks for your reply and the clarification. I fully agree with that.
So I think we have two possible ways to go for now:- If the Suricata-Update 1.2.x series is still considered supported, than we should fix this in 1.2.8 and throw a warning, that with upgrading to the 1.3.x series this "wrong" references will cause errors.
- If only Suricata-Update 1.3.x series is considered supported, this "bug" was implicitly fixed by 1.3.x. We could append a reference to this redmine ticket in the changelog, so users will be able to find this ticket if they notice the changed behaviour (like we do).
I doubt that a fix in 1.2.x will reach many users and is thus not worth the time, so I would suggest not considering this a bug any more and at best would add some hint in the 1.3.x changelog or somewhere else.
(E.g. in Debian Suricata-Update 1.2.x is oldstable/bookworm and will not getting updates any more.)
Thanks!