https://redmine.openinfosecfoundation.org/https://redmine.openinfosecfoundation.org/favicon.ico?17011170022021-02-20T08:07:10ZOpen Information Security FoundationSuricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=193632021-02-20T08:07:10ZGatewatcher Dev Team
<ul></ul><p>Hi,</p>
<p>Just an quick update from our side: we have been able to confirm that the issue is still present in the latest stable version of Suricata (6.0.1 at the time of writing). The fileinfo event continues to indicate that the file was not stored even though it effectively was stored.</p>
<p>We have heard feedback from production networks where these false reports represent north of 25% of files stored by Suricata. This problem is significant and we are a bit abashed that this bug report has remained unanswered and unaddressed, as accuracy is probably one of the most desirable attributes of an IDS.</p>
<p>We have not been able to come-up with a fix of our own for this issue, and help would be very much appreciated.</p>
<p>Thank you.</p>
<p>Cheers,<br />Florian Maury</p> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=195022021-03-06T07:58:02ZVictor Julienvictor@inliniac.net
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>Jeff Lucovsky</i></li><li><strong>Target version</strong> set to <i>7.0.0-beta1</i></li><li><strong>Label</strong> <i>Needs backport to 5.0, Needs backport to 6.0</i> added</li></ul> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=195112021-03-06T18:21:57ZJeff Lucovsky
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>In Progress</i></li></ul><p>Removed errant post.</p>
<p>I was able to reproduce the issue -- sorry for the confusion.</p> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=195122021-03-06T18:22:14ZJeff Lucovsky
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-1 status-6 priority-4 priority-default closed" href="/issues/4382">Bug #4382</a>: fileinfo "stored: false" even if the file is kept on disk</i> added</li></ul> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=195142021-03-06T18:22:36ZJeff Lucovsky
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-1 status-5 priority-4 priority-default closed" href="/issues/4383">Bug #4383</a>: fileinfo "stored: false" even if the file is kept on disk</i> added</li></ul> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=195162021-03-08T11:20:10ZGatewatcher Dev Team
<ul><li><strong>File</strong> <a href="/attachments/2273">reproduction.tgz</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2273/reproduction.tgz">reproduction.tgz</a> added</li></ul><p>Hi,</p>
<p>Thank you for trying to reproduce the bug.</p>
<p>It is really weird that you cannot; it might be related to some config options that you have activated.</p>
<p>The attached archive contains a Dockerfile and all of the resources necessary to reproduce the bug. The dockerfile checks out from github, and builds from source. The config file is the one provided by the Debian package on the OISF PPA, with filestore enabled and the paths modified to make it work.</p>
<p>You can just run the following commands:</p>
<p>```<br />podman build -t suricata5 --build-arg SURIBRANCH=master-5.0.x .<br />podman run suricata5<br />podman build -t suricata6 --build-arg SURIBRANCH=master-6.0.x .<br />podman run suricata6<br />```</p>
<p>I get the following output:</p>
<p>```<br />8/3/2021 -- 11:16:07 - <Notice> - This is Suricata version 5.0.6 RELEASE running in USER mode<br />8/3/2021 -- 11:16:07 - <Notice> - JsonRdpLog logger not enabled: protocol rdp is disabled<br />8/3/2021 -- 11:16:07 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - No output module named eve-log.dcerpc<br />8/3/2021 -- 11:16:07 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - No output module named eve-log.rfb<br />8/3/2021 -- 11:16:07 - <Notice> - JsonSIPLog logger not enabled: protocol sip is disabled<br />8/3/2021 -- 11:16:07 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - No output module named eve-log.mqtt<br />8/3/2021 -- 11:16:07 - <Notice> - all 13 packet processing threads, 4 management threads initialized, engine started.<br />8/3/2021 -- 11:16:07 - <Notice> - Signal Received. Stopping engine.<br />8/3/2021 -- 11:16:07 - <Notice> - Pcap-file module read 1 files, 22 packets, 11749 bytes
{<br /> "timestamp": "2019-09-30T09:16:05.585275+0000",<br /> "flow_id": 1029515603471061,<br /> "pcap_cnt": 17,<br /> "event_type": "fileinfo",<br /> "src_ip": "192.168.0.2",<br /> "src_port": 80,<br /> "dest_ip": "192.168.0.1",<br /> "dest_port": 36656,<br /> "proto": "TCP",<br /> "http": {<br /> "hostname": "192.168.0.2",<br /> "url": "/467659.pdf",<br /> "http_content_type": "application/pdf",<br /> "http_method": "GET",<br /> "protocol": "HTTP/1.1",<br /> "status": 200,<br /> "length": 9952<br /> },<br /> "app_proto": "http",<br /> "fileinfo": {<br /> "filename": "/467659.pdf",<br /> "sid": [],<br /> "gaps": false,<br /> "state": "CLOSED",<br /> "sha256": "02f43016d07812f881dc1ccee724f95682016ff00c7ee6b2c856d4d693ce3fa5",<br /> "stored": false,<br /> "size": 9952,<br /> "tx_id": 0<br /> }<br />}<br /> 12324026 12 <del>rw-r--r-</del> 1 root root 9952 Mar 8 11:16 filestore/02/02f43016d07812f881dc1ccee724f95682016ff00c7ee6b2c856d4d693ce3fa5</p>
<p>```</p>
<p>As you can see, I have a file in the filestore but the fileinfo states `"stored": false`</p>
<p>Please let us know if you manage to reproduce the issue with the provided dockerfile.</p>
<p>Thank you.</p>
<p>Cheers,<br />Florian Maury</p> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=196442021-03-29T13:28:20ZVictor Julienvictor@inliniac.net
<ul><li><strong>Assignee</strong> changed from <i>Jeff Lucovsky</i> to <i>Victor Julien</i></li></ul><p>Work in progress here <a class="external" href="https://github.com/OISF/suricata/pull/5999">https://github.com/OISF/suricata/pull/5999</a></p> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=199922021-05-24T14:55:09ZGatewatcher Dev Team
<ul></ul><p>Hi,</p>
<p>We would like to thank you for the work that has been done regarding this issue. It seems like the source cause has been a real pain to tackle, but that you have made good progress.</p>
<p>We would like to know if there is any chance for the linked PR to be merged in the near future. Are there any roadblocks that we are missing to see preventing the PR from being merged? And if so, could you please provide instructions to help you merge this PR?</p>
<p>Thank you.</p>
<p>Regards,<br />Florian Maury</p> Suricata - Bug #3703: fileinfo "stored: false" even if the file is kept on diskhttps://redmine.openinfosecfoundation.org/issues/3703?journal_id=206602021-09-15T07:29:31ZVictor Julienvictor@inliniac.net
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Closed</i></li></ul><p><a class="external" href="https://github.com/OISF/suricata/pull/5999">https://github.com/OISF/suricata/pull/5999</a> through <a class="external" href="https://github.com/OISF/suricata/pull/6324">https://github.com/OISF/suricata/pull/6324</a></p>