Project

General

Profile

Actions

Bug #5486

open

Ethernet metadata is missing for some protocols or parts of a protocol

Added by Andreas Herz about 2 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

We discovered that the `ether` output was missing on some traffic, with the pcap 2019-02-15-Emotet-with-IcedID-and-Trickbot.pcap from traffic malware analysis we were able to strip it down to some single flow cases.

To reproduce it, simply run one of the 3 pcaps with Suricata 6.0.6 and the `-r` or use a `dummy` interface.

In the 154.pcap we can see that it is empty for the http event but not for the flow and it it's completely missing for fileinfo, http output as an example:

{
  "timestamp": "2019-02-15T19:29:41.539954+0100",
  "flow_id": 183210505223474,
  "event_type": "http",
  "src_ip": "172.16.10.97",
  "src_port": 49910,
  "dest_ip": "70.184.86.103",
  "dest_port": 8080,
  "proto": "TCP",
  "tx_id": 0,
  "ether": {},
  "http": {
    "hostname": "70.184.86.103",
    "http_port": 8080,
    "url": "/",
    "http_user_agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
    "http_content_type": "text/html",
    "http_method": "GET",
    "protocol": "HTTP/1.1",
    "status": 200,
    "length": 135041
  }
}

In the 136.pcap with smb traffic we can see it is seen in the flow but not on all packets. See it in the first example to be present and in the second emtpy:

{
  "timestamp": "2019-02-15T19:26:50.118392+0100",
  "flow_id": 556954253130233,
  "pcap_cnt": 63,
  "event_type": "anomaly",
  "src_ip": "172.16.10.2",
  "src_port": 445,
  "dest_ip": "172.16.10.97",
  "dest_port": 49892,
  "proto": "TCP",
  "tx_id": 16,
  "ether": {
    "src_mac": "00:08:02:1c:47:ae",
    "dest_mac": "a4:1f:72:c2:09:6a" 
  },
  "anomaly": {
    "app_proto": "smb",
    "type": "applayer",
    "event": "malformed_data",
    "layer": "proto_parser" 
  }
}
{
  "timestamp": "2019-02-15T19:26:49.915961+0100",
  "flow_id": 556954253130233,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 9,
    "dialect": "NT LM 0.12",
    "command": "161",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {}
}

In the 126.pcap with dcerpc ether is missing from the event type, but seen in the flow event:

{
  "timestamp": "2019-02-15T19:26:38.567904+0100",
  "flow_id": 1709083524481263,
  "pcap_cnt": 7,
  "event_type": "dcerpc",
  "src_ip": "172.16.10.97",
  "src_port": 49801,
  "dest_ip": "172.16.10.2",
  "dest_port": 135,
  "proto": "TCP",
  "dcerpc": {
    "request": "REQUEST",
    "req": {
      "opnum": 3,
      "frag_cnt": 1,
      "stub_data_size": 144
    },
    "response": "RESPONSE",
    "res": {
      "frag_cnt": 1,
      "stub_data_size": 252
    },
    "call_id": 2,
    "rpc_version": "5.0" 
  }
}
{
  "timestamp": "2019-02-15T19:26:38.344303+0100",
  "flow_id": 1709083524481263,
  "event_type": "flow",
  "src_ip": "172.16.10.97",
  "src_port": 49801,
  "dest_ip": "172.16.10.2",
  "dest_port": 135,
  "proto": "TCP",
  "app_proto": "dcerpc",
  "flow": {
    "pkts_toserver": 6,
    "pkts_toclient": 5,
    "bytes_toserver": 664,
    "bytes_toclient": 658,
    "start": "2019-02-15T19:26:38.344303+0100",
    "end": "2019-02-15T19:26:53.367110+0100",
    "age": 15,
    "state": "closed",
    "reason": "shutdown",
    "alerted": false
  },
  "ether": {
    "dest_macs": [
      "a4:1f:72:c2:09:6a" 
    ],
    "src_macs": [
      "00:08:02:1c:47:ae" 
    ]
  },
  "tcp": {
    "tcp_flags": "1b",
    "tcp_flags_ts": "1b",
    "tcp_flags_tc": "1b",
    "syn": true,
    "fin": true,
    "psh": true,
    "ack": true,
    "state": "closed" 
  }
}


Files

126.pcap (1.49 KB) 126.pcap Andreas Herz, 08/08/2022 11:44 AM
136.pcap (28.6 KB) 136.pcap Andreas Herz, 08/08/2022 11:44 AM
154.pcap (147 KB) 154.pcap Andreas Herz, 08/08/2022 11:44 AM
Actions #1

Updated by Andreas Herz about 2 months ago

more smb example:

{
  "timestamp": "2019-02-15T19:26:50.117953+0100",
  "flow_id": 556954253130233,
  "pcap_cnt": 52,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 14,
    "dialect": "NT LM 0.12",
    "command": "SMB1_COMMAND_NT_TRANS",
    "status": "STATUS_SUCCESS",
    "status_code": "0x0",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {
    "src_mac": "00:08:02:1c:47:ae",
    "dest_mac": "a4:1f:72:c2:09:6a" 
  }
}
{
  "timestamp": "2019-02-15T19:26:49.915961+0100",
  "flow_id": 556954253130233,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 9,
    "dialect": "NT LM 0.12",
    "command": "161",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {}
}

Actions #2

Updated by Victor Julien about 2 months ago

Can you turn these examples in to SV tests?

Actions #3

Updated by Andreas Herz about 2 months ago

Of course, will do that

Actions #4

Updated by Andreas Herz about 2 months ago

So we narrowed it down to two issues, the first one is rather simple:

The `EveAddCommonOptions` is missing for dcerpc, which is also true for mqtt, quic, pgsql in that version. But this is fixed due to the changes in version 7.0.0. So now all protocols should add the EVE additional information like ethernet and community id (if enabled).

The second one for the smb and http pcap are the same issue with 7.0 but is a bit more complicated. The packets all pass `DecodeEthernet` correctly and the ethernet data is there but once they need to be written, some of those run into `CreateJSONEther` with `p->ethh` being `NULL` which shouldn't be since it was present at the decoding state.

Actions

Also available in: Atom PDF