Project

General

Profile

Actions

Bug #6388

closed

files: do not store file if we do not have all the content (master6 only)

Added by Philippe Antoine about 1 year ago. Updated 11 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
Affected Versions:
Effort:
Difficulty:
Label:

Description

On master6, there can be some scenarios that lead to storing a file, without having its content.
This can be checked by checking the hash of the file against its name.
This does not appear on master7 mostly as FileAppendDataDo also checks for g_detect_disabled to not track the content. (so having both flags FILE_STORE | FILE_NOSTORE is not a big issue, and code has been rewritten removing scenarios leading to setting both flags)
But FileCloseFilePtr still does not call AppendData in the case ff->flags & FILE_NOSTORE so there may be something to do there (plus adding debug validations)

Scenarios are :
- Keyword filestore:both,flow with a non-default scope, will try to store all the live files in the flow.
But if this happens on a pipelined HTTP1 request, while the previous HTTP response is being received, response whose file was judged nostore, this reponse file will end up being stored, but not with its full contents.
In master7, this code/behavior has been changed so that only files opened after detection has triggered will be stored

- Rule reload
If a rule reload removes some rule storing a file, but a file file was being stored thanks to that rule, we lose the final content, and end up storing the file without its full contents

- Hypothetic call to FileCloseFilePtr. with flags FILE_TRUNCATED and FILE_NOSTORE
I do not see any call graph to this point.
I do not think setting ff->flags |= FILE_NOSTORE; has any effect
Unset FILE_STORE seems better suited to the intent as that is the one flag checked in OutputFiledataLogFfc

- Another problem appears with using keyword filestore:request,flow with a directional scope
There is a typo in the flags used ts versus tc
In master7, this code/behavior has been changed so that only files opened after detection has triggered will be stored

- Also if I have 65536 signatures with file stores in the same signature group head, some u16 overflows silently, and detection does not run as expected


Subtasks 2 (0 open2 closed)

Bug #6391: file: still store if rule reload removes the rule storing the fileRejectedActions
Bug #6392: detect/filestore: respect directionality for filestore:request,flow RejectedActions
Actions #1

Updated by Philippe Antoine about 1 year ago

Also if I have 65536 signatures with file stores in the same signature group head, some u16 overflows silently, and detection does not run as expected

That is present in master7

Actions #2

Updated by Philippe Antoine about 1 year ago

  • Status changed from New to In Review
Actions #3

Updated by Victor Julien about 1 year ago

Please create separate tickets for each of the issues.

Also, please add SV tests where possible.

Actions #4

Updated by Philippe Antoine about 1 year ago

  • Subtask #6390 added
Actions #5

Updated by Philippe Antoine about 1 year ago

  • Subtask #6391 added
Actions #6

Updated by Philippe Antoine about 1 year ago

  • Subtask #6392 added
Actions #7

Updated by Philippe Antoine about 1 year ago

  • Subtask #6393 added
Actions #8

Updated by Victor Julien about 1 year ago

  • Target version changed from 6.0.15 to 6.0.16
Actions #9

Updated by Victor Julien 11 months ago

  • Target version changed from 6.0.16 to 6.0.17
Actions #10

Updated by Victor Julien 11 months ago

  • Subtask deleted (#6390)
Actions #11

Updated by Victor Julien 11 months ago

  • Subtask deleted (#6393)
Actions #12

Updated by Victor Julien 11 months ago

  • Status changed from In Review to Rejected
  • Assignee deleted (Philippe Antoine)
  • Target version deleted (6.0.17)

Rejecting as 6.0.x is now in extended support and EOL in 6 months.

Actions

Also available in: Atom PDF