Feature #2147: fileinfo: sha1 hash not logged if state == TRUNCATED - Suricata - Open Information Security Foundation

Actions

Copy link

Feature #2147

open

fileinfo: sha1 hash not logged if state == TRUNCATED

Added by Fanny Dwargee over 7 years ago. Updated about 1 year ago.

Status:

New

Priority:

Normal

Assignee:

Fanny Dwargee

Target version:

TBD

Effort:

Difficulty:

Label:

Description

Tested on Suricata version "4.0.0-beta1 RELEASE" with configure options: "--enable-unix-socket"

Find attached pcap file

Related suricata.yaml sections:

    HOME_NET: "[192.168.1.0/24]" 
    DNS_SERVERS: "$HOME_NET" 

  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve.json
      types:
        - alert:
            http: yes
            tls:  yes
            ssh:  no
            smtp: no
            dnp3: no
            vars: no
            tagged-packets: yes
            xff:
              enabled: no
      - dns:
            query: yes
            answer: yes
      - tls:
      - files:
            force-magic: yes
            force-hash: [sha1]
      - flow

   - file-store:
      enabled: yes
      log-dir: ids-files
      force-magic: no
      force-filestore: yes
      stream-depth: 0
      write-meta: no

   - file-log:
      enabled: no

Fileinfo from eve json file with sha1 logged (state CLOSED), manually added carriage returns:

{"timestamp":"2016-05-09T15:16:28.822021+0200","flow_id":589022781737826,"pcap_cnt":1845,
"event_type":"fileinfo",
"src_ip":"192.168.1.14","src_port":1068,"dest_ip":"91.223.216.67","dest_port":80,"proto":"TCP","http":{"hostname":"a6281279.yolox.net","url":"/gate.php","http_user_agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)","http_content_type":"text/html","http_method":"POST","protocol":"HTTP/1.1","status":405,"length":1705},"app_proto":"http",
"fileinfo":{
             "filename":"/gate.php",
             "magic":"data",
             "state":"CLOSED",
             "sha1":"99e30409ff5804e3ab4ca2bb584d55433243881e",
             "stored":true,
             "file_id":19,
             "size":40,
             "tx_id":0
           }
}

Fileinfo from eve json file without sha1 logged (state TRUNCATED), manually added carriage returns:

{"timestamp":"2016-05-09T15:15:08.916699+0200","flow_id":117276453454472,"pcap_cnt":106,
"event_type":"fileinfo",
"src_ip":"192.168.1.14","src_port":1036,"dest_ip":"74.125.34.46","dest_port":80,"proto":"TCP","http":{"hostname":"www.virustotal.com","url":"/vtapi/v2/file/scan","http_user_agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)","http_method":"POST","protocol":"HTTP/1.1","length":0},"app_proto":"http",
"fileinfo":{
             "filename":"file.exe",
             "magic":"PE32 executable (GUI) Intel 80386, for MS Windows",
             "state":"TRUNCATED",
             "stored":true,
             "file_id":1,
             "size":20109,
             "tx_id":0
           }
}

I don't know if that's a bug or a "feature" but IMHO the file hash is a must apart from the fact of being "TRUNCATED" or "CLOSED".

Regards

Files

malware.pcap (4.93 MB) malware.pcap

Malware traffic

Fanny Dwargee, 06/14/2017 11:44 AM

Actions

Copy link

Updated by Victor Julien over 7 years ago

This is intentional. If we know the file is not complete we don't hash it.

Actions

Copy link

Updated by Fanny Dwargee over 7 years ago

I really understand your point of view but I don't share it, for me the fileinfo->hash is (must be) the hash of something written to disk while the fileinfo->state marks the state of that file, if that hash is important or not to the user will be relevant to the user.

If the fileinfo->state didn't exist it would be a problem but in fact it exists so... but that's just my opinion. ;)

Thank you anyway Victor

Actions

Copy link

Updated by Victor Julien over 7 years ago

Priority changed from High to Normal

Actions

Copy link

Updated by Andreas Herz over 7 years ago

Assignee set to Fanny Dwargee
Target version set to TBD

Is this something we might want to have as an option to configure the behavior?

Actions

Copy link

Updated by Fanny Dwargee over 7 years ago

Andreas Herz wrote:

Is this something we might want to have as an option to configure the behavior?

IMHO it would be very great.

I currently make an internal patch for adding that behaviour to the current 4.0.0-beta1 (that lacks a configurable option, btw) and, sincerely, I don't feel like having to support patching the outcoming versions. :)

Actions

Copy link

Updated by Fanny Dwargee over 7 years ago

Fanny Dwargee wrote:

I currently make an internal patch for adding that behaviour to the current 4.0.0-beta1 (that lacks a configurable option, btw) and, sincerely, I don't feel like having to support patching the outcoming versions. :)

"made", not "make" :)

Actions

Copy link

Updated by Andreas Herz over 7 years ago

I guess that's a patch that doesn't need your support for the future if it's well done, so don't bother to submit a pull request :)

Actions

Copy link

Updated by Fanny Dwargee over 7 years ago

Andreas Herz wrote:

I guess that's a patch that doesn't need your support for the future if it's well done, so don't bother to submit a pull request :)

There's a PR queued on GitHub adding that feature with a new option named 'force-hash-truncated', the PR is https://github.com/inliniac/suricata/pull/2823

Would you mind to take a look?

Hope this helps in any way or another :)

Actions

Copy link