Project

General

Profile

Actions

Feature #2147

open

fileinfo: sha1 hash not logged if state == TRUNCATED

Added by Fanny Dwargee almost 7 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

Tested on Suricata version "4.0.0-beta1 RELEASE" with configure options: "--enable-unix-socket"

Find attached pcap file

Related suricata.yaml sections:

    HOME_NET: "[192.168.1.0/24]" 
    DNS_SERVERS: "$HOME_NET" 

  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve.json
      types:
        - alert:
            http: yes
            tls:  yes
            ssh:  no
            smtp: no
            dnp3: no
            vars: no
            tagged-packets: yes
            xff:
              enabled: no
      - dns:
            query: yes
            answer: yes
      - tls:
      - files:
            force-magic: yes
            force-hash: [sha1]
      - flow

   - file-store:
      enabled: yes
      log-dir: ids-files
      force-magic: no
      force-filestore: yes
      stream-depth: 0
      write-meta: no

   - file-log:
      enabled: no

Fileinfo from eve json file with sha1 logged (state CLOSED), manually added carriage returns:

{"timestamp":"2016-05-09T15:16:28.822021+0200","flow_id":589022781737826,"pcap_cnt":1845,
"event_type":"fileinfo",
"src_ip":"192.168.1.14","src_port":1068,"dest_ip":"91.223.216.67","dest_port":80,"proto":"TCP","http":{"hostname":"a6281279.yolox.net","url":"/gate.php","http_user_agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)","http_content_type":"text/html","http_method":"POST","protocol":"HTTP/1.1","status":405,"length":1705},"app_proto":"http",
"fileinfo":{
             "filename":"/gate.php",
             "magic":"data",
             "state":"CLOSED",
             "sha1":"99e30409ff5804e3ab4ca2bb584d55433243881e",
             "stored":true,
             "file_id":19,
             "size":40,
             "tx_id":0
           }
}

Fileinfo from eve json file without sha1 logged (state TRUNCATED), manually added carriage returns:

{"timestamp":"2016-05-09T15:15:08.916699+0200","flow_id":117276453454472,"pcap_cnt":106,
"event_type":"fileinfo",
"src_ip":"192.168.1.14","src_port":1036,"dest_ip":"74.125.34.46","dest_port":80,"proto":"TCP","http":{"hostname":"www.virustotal.com","url":"/vtapi/v2/file/scan","http_user_agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)","http_method":"POST","protocol":"HTTP/1.1","length":0},"app_proto":"http",
"fileinfo":{
             "filename":"file.exe",
             "magic":"PE32 executable (GUI) Intel 80386, for MS Windows",
             "state":"TRUNCATED",
             "stored":true,
             "file_id":1,
             "size":20109,
             "tx_id":0
           }
}

I don't know if that's a bug or a "feature" but IMHO the file hash is a must apart from the fact of being "TRUNCATED" or "CLOSED".

Regards


Files

malware.pcap (4.93 MB) malware.pcap Malware traffic Fanny Dwargee, 06/14/2017 11:44 AM
Actions #1

Updated by Victor Julien almost 7 years ago

This is intentional. If we know the file is not complete we don't hash it.

Actions #2

Updated by Fanny Dwargee almost 7 years ago

I really understand your point of view but I don't share it, for me the fileinfo->hash is (must be) the hash of something written to disk while the fileinfo->state marks the state of that file, if that hash is important or not to the user will be relevant to the user.

If the fileinfo->state didn't exist it would be a problem but in fact it exists so... but that's just my opinion. ;)

Thank you anyway Victor

Actions #3

Updated by Victor Julien almost 7 years ago

  • Priority changed from High to Normal
Actions #4

Updated by Andreas Herz almost 7 years ago

  • Assignee set to Fanny Dwargee
  • Target version set to TBD

Is this something we might want to have as an option to configure the behavior?

Actions #5

Updated by Fanny Dwargee almost 7 years ago

Andreas Herz wrote:

Is this something we might want to have as an option to configure the behavior?

IMHO it would be very great.

I currently make an internal patch for adding that behaviour to the current 4.0.0-beta1 (that lacks a configurable option, btw) and, sincerely, I don't feel like having to support patching the outcoming versions. :)

Actions #6

Updated by Fanny Dwargee almost 7 years ago

Fanny Dwargee wrote:

I currently make an internal patch for adding that behaviour to the current 4.0.0-beta1 (that lacks a configurable option, btw) and, sincerely, I don't feel like having to support patching the outcoming versions. :)

"made", not "make" :)

Actions #7

Updated by Andreas Herz over 6 years ago

I guess that's a patch that doesn't need your support for the future if it's well done, so don't bother to submit a pull request :)

Actions #8

Updated by Fanny Dwargee over 6 years ago

Andreas Herz wrote:

I guess that's a patch that doesn't need your support for the future if it's well done, so don't bother to submit a pull request :)

There's a PR queued on GitHub adding that feature with a new option named 'force-hash-truncated', the PR is https://github.com/inliniac/suricata/pull/2823

Would you mind to take a look?

Hope this helps in any way or another :)

Actions #9

Updated by Andreas Herz almost 5 years ago

The last PR I could find was https://github.com/OISF/suricata/pull/2898 so how should we proceed?

Actions #10

Updated by Fanny Dwargee almost 5 years ago

If you're asking me I have to say that the aforementioned PR was closed by Victor Julien with no clear reason (at least for me).

Actions #12

Updated by Philippe Antoine 5 months ago

  • Tracker changed from Bug to Feature
Actions

Also available in: Atom PDF