Bug #6388
closedfiles: do not store file if we do not have all the content (master6 only)
Description
On master6, there can be some scenarios that lead to storing a file, without having its content.
This can be checked by checking the hash of the file against its name.
This does not appear on master7 mostly as FileAppendDataDo
also checks for g_detect_disabled
to not track the content. (so having both flags FILE_STORE | FILE_NOSTORE
is not a big issue, and code has been rewritten removing scenarios leading to setting both flags)
But FileCloseFilePtr
still does not call AppendData
in the case ff->flags & FILE_NOSTORE
so there may be something to do there (plus adding debug validations)
Scenarios are :
- Keyword filestore:both,flow
with a non-default scope, will try to store all the live files in the flow.
But if this happens on a pipelined HTTP1 request, while the previous HTTP response is being received, response whose file was judged nostore, this reponse file will end up being stored, but not with its full contents.
In master7, this code/behavior has been changed so that only files opened after detection has triggered will be stored
- Rule reload
If a rule reload removes some rule storing a file, but a file file was being stored thanks to that rule, we lose the final content, and end up storing the file without its full contents
- Hypothetic call to FileCloseFilePtr
. with flags FILE_TRUNCATED
and FILE_NOSTORE
I do not see any call graph to this point.
I do not think setting ff->flags |= FILE_NOSTORE;
has any effect
Unset FILE_STORE
seems better suited to the intent as that is the one flag checked in OutputFiledataLogFfc
- Another problem appears with using keyword filestore:request,flow
with a directional scope
There is a typo in the flags used ts
versus tc
In master7, this code/behavior has been changed so that only files opened after detection has triggered will be stored
- Also if I have 65536 signatures with file stores in the same signature group head, some u16 overflows silently, and detection does not run as expected
Updated by Philippe Antoine about 1 year ago
Also if I have 65536 signatures with file stores in the same signature group head, some u16 overflows silently, and detection does not run as expected
That is present in master7
Updated by Philippe Antoine about 1 year ago
- Status changed from New to In Review
Updated by Victor Julien about 1 year ago
Please create separate tickets for each of the issues.
Also, please add SV tests where possible.
Updated by Victor Julien about 1 year ago
- Target version changed from 6.0.15 to 6.0.16
Updated by Victor Julien 11 months ago
- Target version changed from 6.0.16 to 6.0.17
Updated by Victor Julien 11 months ago
- Status changed from In Review to Rejected
- Assignee deleted (
Philippe Antoine) - Target version deleted (
6.0.17)
Rejecting as 6.0.x is now in extended support and EOL in 6 months.