Suricata » Suricata-Update

Hi Suricata Team,

we've noticed some inconsistent behaviour between versions while upgrading Suricata-Update from 1.2.7 (Debian Bullseye-Backports) to 1.3.4 (Debian Bookworm-Backports). We can reproduce this by the attached minimal example rules.
Given a directory structure as follows:

# tar -tf local-rules.tar.gz
rules/
rules/subfolder/
rules/rules1.rules
rules/hashes1.md5
rules/subfolder/rules2.rules
rules/subfolder/hashes2.md5
rules/subfolder/hashes3.md5
rules/subfolder/hashes4.md5

The hash-files are all identical for this demo.
The rule-files look like this:

# cat rules1.rules 
alert http any any -> any any (msg:"Rule in mainfolder, hashfile in mainfolder, filename reference without subfolder"; filemd5:hashes1.md5; sid:1000001;)
alert http any any -> any any (msg:"Rule in mainfolder, hashfile in subfolder, filename reference with subfolder"; filemd5:subfolder/hashes2.md5; sid:1000002;)

# cat subfolder/rules2.rules 
alert http any any -> any any (msg:"Rule in subfolder, hashfile in subfolder, filename reference without subfolder"; filemd5:hashes3.md5; sid:1000003;)
alert http any any -> any any (msg:"Rule in subfolder, hashfile in subfolder, filename reference with subfolder"; filemd5:subfolder/hashes4.md5; sid:1000004;)

In Suricata-Update 1.2.7, rule 1000003 fails ( <Error> -- filemd5 file hashes3.md5 was not found ), so we built all our rules like rule 1000004, with the 'subfolder/' in the filemd5.
Loading exactly the same local-rules.tar.gz in Suricata-Update 1.3.4, rule 1000003 is OK now, but rule 1000004 fails ( <Error> -- filemd5 file subfolder/hashes4.md5 was not found ). This is how we found this behaviour.

After some troubleshooting in the Python code, I suppose this behaviour is caused by the changed file-handling because of the work done on using hashes in the filenames. I noticed that for 1.3.4 the variable is source_filename=54f36aa263627c6935edd30702d67e76/rules/subfolder/subfolder/hashes4.md5 , having a double 'subfolder/subfolder' in it.

I can try to do a PR fixing this, but before I'd like to clarify, what the intended behaviour of Suricata-Update actually is.
Shall a filename-reference for a rule in a subfolder be relative to the rule itself, or to the final suricata.rules in the parent directory, or shall both cases be supported? (I know that in any cases the final rules have to be loaded without any errors in Suricata.)

Thanks for your answers and for your help.
Andreas