Project

General

Profile

Actions

Bug #5486

open

Ethernet metadata is missing for some protocols or parts of a protocol

Added by Andreas Herz over 2 years ago. Updated 7 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

We discovered that the `ether` output was missing on some traffic, with the pcap 2019-02-15-Emotet-with-IcedID-and-Trickbot.pcap from traffic malware analysis we were able to strip it down to some single flow cases.

To reproduce it, simply run one of the 3 pcaps with Suricata 6.0.6 and the `-r` or use a `dummy` interface.

In the 154.pcap we can see that it is empty for the http event but not for the flow and it it's completely missing for fileinfo, http output as an example:

{
  "timestamp": "2019-02-15T19:29:41.539954+0100",
  "flow_id": 183210505223474,
  "event_type": "http",
  "src_ip": "172.16.10.97",
  "src_port": 49910,
  "dest_ip": "70.184.86.103",
  "dest_port": 8080,
  "proto": "TCP",
  "tx_id": 0,
  "ether": {},
  "http": {
    "hostname": "70.184.86.103",
    "http_port": 8080,
    "url": "/",
    "http_user_agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
    "http_content_type": "text/html",
    "http_method": "GET",
    "protocol": "HTTP/1.1",
    "status": 200,
    "length": 135041
  }
}

In the 136.pcap with smb traffic we can see it is seen in the flow but not on all packets. See it in the first example to be present and in the second emtpy:

{
  "timestamp": "2019-02-15T19:26:50.118392+0100",
  "flow_id": 556954253130233,
  "pcap_cnt": 63,
  "event_type": "anomaly",
  "src_ip": "172.16.10.2",
  "src_port": 445,
  "dest_ip": "172.16.10.97",
  "dest_port": 49892,
  "proto": "TCP",
  "tx_id": 16,
  "ether": {
    "src_mac": "00:08:02:1c:47:ae",
    "dest_mac": "a4:1f:72:c2:09:6a" 
  },
  "anomaly": {
    "app_proto": "smb",
    "type": "applayer",
    "event": "malformed_data",
    "layer": "proto_parser" 
  }
}
{
  "timestamp": "2019-02-15T19:26:49.915961+0100",
  "flow_id": 556954253130233,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 9,
    "dialect": "NT LM 0.12",
    "command": "161",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {}
}

In the 126.pcap with dcerpc ether is missing from the event type, but seen in the flow event:

{
  "timestamp": "2019-02-15T19:26:38.567904+0100",
  "flow_id": 1709083524481263,
  "pcap_cnt": 7,
  "event_type": "dcerpc",
  "src_ip": "172.16.10.97",
  "src_port": 49801,
  "dest_ip": "172.16.10.2",
  "dest_port": 135,
  "proto": "TCP",
  "dcerpc": {
    "request": "REQUEST",
    "req": {
      "opnum": 3,
      "frag_cnt": 1,
      "stub_data_size": 144
    },
    "response": "RESPONSE",
    "res": {
      "frag_cnt": 1,
      "stub_data_size": 252
    },
    "call_id": 2,
    "rpc_version": "5.0" 
  }
}
{
  "timestamp": "2019-02-15T19:26:38.344303+0100",
  "flow_id": 1709083524481263,
  "event_type": "flow",
  "src_ip": "172.16.10.97",
  "src_port": 49801,
  "dest_ip": "172.16.10.2",
  "dest_port": 135,
  "proto": "TCP",
  "app_proto": "dcerpc",
  "flow": {
    "pkts_toserver": 6,
    "pkts_toclient": 5,
    "bytes_toserver": 664,
    "bytes_toclient": 658,
    "start": "2019-02-15T19:26:38.344303+0100",
    "end": "2019-02-15T19:26:53.367110+0100",
    "age": 15,
    "state": "closed",
    "reason": "shutdown",
    "alerted": false
  },
  "ether": {
    "dest_macs": [
      "a4:1f:72:c2:09:6a" 
    ],
    "src_macs": [
      "00:08:02:1c:47:ae" 
    ]
  },
  "tcp": {
    "tcp_flags": "1b",
    "tcp_flags_ts": "1b",
    "tcp_flags_tc": "1b",
    "syn": true,
    "fin": true,
    "psh": true,
    "ack": true,
    "state": "closed" 
  }
}


Files

126.pcap (1.49 KB) 126.pcap Andreas Herz, 08/08/2022 11:44 AM
136.pcap (28.6 KB) 136.pcap Andreas Herz, 08/08/2022 11:44 AM
154.pcap (147 KB) 154.pcap Andreas Herz, 08/08/2022 11:44 AM
suricata.yaml (73.1 KB) suricata.yaml Raj S, 03/23/2023 12:04 PM
buildinfo.txt (3.87 KB) buildinfo.txt Raj S, 03/23/2023 12:05 PM
Actions #1

Updated by Andreas Herz over 2 years ago

more smb example:

{
  "timestamp": "2019-02-15T19:26:50.117953+0100",
  "flow_id": 556954253130233,
  "pcap_cnt": 52,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 14,
    "dialect": "NT LM 0.12",
    "command": "SMB1_COMMAND_NT_TRANS",
    "status": "STATUS_SUCCESS",
    "status_code": "0x0",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {
    "src_mac": "00:08:02:1c:47:ae",
    "dest_mac": "a4:1f:72:c2:09:6a" 
  }
}
{
  "timestamp": "2019-02-15T19:26:49.915961+0100",
  "flow_id": 556954253130233,
  "event_type": "smb",
  "src_ip": "172.16.10.97",
  "src_port": 49892,
  "dest_ip": "172.16.10.2",
  "dest_port": 445,
  "proto": "TCP",
  "smb": {
    "id": 9,
    "dialect": "NT LM 0.12",
    "command": "161",
    "session_id": 2051,
    "tree_id": 4100
  },
  "ether": {}
}

Actions #2

Updated by Victor Julien over 2 years ago

Can you turn these examples in to SV tests?

Actions #3

Updated by Andreas Herz over 2 years ago

Of course, will do that

Actions #4

Updated by Andreas Herz over 2 years ago

So we narrowed it down to two issues, the first one is rather simple:

The `EveAddCommonOptions` is missing for dcerpc, which is also true for mqtt, quic, pgsql in that version. But this is fixed due to the changes in version 7.0.0. So now all protocols should add the EVE additional information like ethernet and community id (if enabled).

The second one for the smb and http pcap are the same issue with 7.0 but is a bit more complicated. The packets all pass `DecodeEthernet` correctly and the ethernet data is there but once they need to be written, some of those run into `CreateJSONEther` with `p->ethh` being `NULL` which shouldn't be since it was present at the decoding state.

Updated by Raj S almost 2 years ago

The reason for the missing ether is due to FlowFinish() creating two
pseudo packets p1 and p2 by calling FlowForceReassemblyPseudoPacketGet().

1. FlowForceReassemblyPseudoPacketGet() obtains a new packet p via PacketPoolGetPacket(), and calls
FlowForceReassemblyPseudoPacketSetup()

2. FlowForceReassemblyPseudoPacketSetup() then proceeds to populate 'p' by copying attributes from
the flow 'f'. Crucially, it never copies over the EthernetHdr 'ethh' from the flow to initialise 'p->ethh' because
in the context of the flow struct 'ethh' doesn't exist. The fact that ethh doesn't exist in the flow struct does make sense
to me (ithink?) but basically this is why CreateJSONEther() fails to output an ether value because
'p->ethh' evaluates to NULL for the newly created pseudo packets. See output-json.c:836.

To further understand the above, run suricata with

gdb -tui --args suricata/src/.libs/suricata -r 154.pcap -c suricata.yaml

and monitor the following breakpoints

(gdb) info breakpoints
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00000000003f8f1f in CreateJSONEther at output-json.c:836
2       breakpoint     keep y   0x00000000003b3e7c in FlowForceReassemblyPseudoPacketSetup at flow-timeout.c:83
3       breakpoint     keep y   0x00000000003b6010 in FlowFinish at flow-worker.c:121
4       breakpoint     keep y   0x00000000003b604b in FlowFinish at flow-worker.c:128
5       breakpoint     keep y   0x00000000003b60d6 in FlowFinish at flow-worker.c:141

Given all the above,
Using the ugly hack below I can get it to output an ethernet value, but I'm not entirely sure it makes sense or whether it is correct.

diff --git a/src/output-json.c b/src/output-json.c
index f262972..f31e7db 100644
--- a/src/output-json.c
+++ b/src/output-json.c
@@ -803,6 +803,7 @@ int CreateJSONEther(JsonBuilder *js, const Packet *p, const Flow *f)
     /* start new EVE sub-object */
     jb_open_object(js, "ether");
     if (p == NULL) {
+invalid_ethh_try_flowpath:;
         MacSet *ms = NULL;
         /* ensure we have a flow */
         if (unlikely(f == NULL)) {
@@ -838,6 +839,8 @@ int CreateJSONEther(JsonBuilder *js, const Packet *p, const Flow *f)
             uint8_t *dst = p->ethh->eth_dst;
             JSONFormatAndAddMACAddr(js, "src_mac", src, false);
             JSONFormatAndAddMACAddr(js, "dest_mac", dst, false);
+        } else {
+            goto invalid_ethh_try_flowpath;
         }
     }
     jb_close(js);
Actions #6

Updated by Raj S almost 2 years ago

Also note that tunneled traffic exhibits similar behavior.

When TmThreadsSlotVarRun runs the Decode* routines to parse a packet and encounters a
tunnel packet, it runs DecodeGRE().

It then calls PacketTunnelPktSetup() to parse the inner packet and calls PacketEnqueueNoLock().
In doing so, the newly enqueued packet has no p->ethh.

This causes CreateJSONEther() to fail.

But in this case, I take it that we can't do anything about it given the nature of tunneling?
Or does it still make sense to copy over the ethh from the parent packet?

Actions #7

Updated by Juliana Fajardini Reichow over 1 year ago

  • Target version changed from TBD to 6.0.13
Actions #8

Updated by Juliana Fajardini Reichow over 1 year ago

  • Target version changed from 6.0.13 to 8.0.0-beta1

Hey @Sascha Steinbiss would you like to take this one?

Actions #9

Updated by Sascha Steinbiss over 1 year ago

Juliana Fajardini Reichow wrote in #note-8:

Hey @Sascha Steinbiss would you like to take this one?

Hmm, I am not sure I can currently dedicate the appropriate time to this right now. But it sounds like an attractive learning opportunity to dive into the way packets are passed through Suricata. So... if this needs to be done in time for a release, I'd rather leave it to someone else. If there is time, I might take it.

Actions #10

Updated by Juliana Fajardini Reichow over 1 year ago

The current timeline for 8.0.0-beta1 is June 4th, would that be doable considering what you've said? If you think it is, we'd be happy to have you work in it! :)

Actions #11

Updated by Sascha Steinbiss over 1 year ago

Unlikely, sorry. I'd rather focus on #5912 as I don't have to dive into unknown code to fix it (and I also kind of told Andreas that I would do it ;))

Actions #12

Updated by Juliana Fajardini Reichow over 1 year ago

Sascha Steinbiss wrote in #note-11:

Unlikely, sorry. I'd rather focus on #5912 as I don't have to dive into unknown code to fix it (and I also kind of told Andreas that I would do it ;))

That's fair! Thanks for working on the other task, and for being responsible about the time you have at the moment. :)

Actions #13

Updated by Juliana Fajardini Reichow over 1 year ago

Sascha Steinbiss wrote in #note-11:

Unlikely, sorry. I'd rather focus on #5912 as I don't have to dive into unknown code to fix it (and I also kind of told Andreas that I would do it ;))

Hey, I messed up when mentioning the release date for 8.0.0-beta1! It's for 2024! Not put pressure in changing your answer, but because my prior message doesn't make a lot of sense, without that part >__<' Sorry about this confusion...

Actions #14

Updated by Joyce Yu 7 months ago

Hi, is anyone working on this ticket? If not, may I pick it up?

Actions #15

Updated by Sascha Steinbiss 7 months ago ยท Edited

At least I am not. I'd say go for it? :)

Actions #16

Updated by Juliana Fajardini Reichow 7 months ago

Joyce Yu wrote in #note-14:

Hi, is anyone working on this ticket? If not, may I pick it up?

What Sascha said, and thanks for contributing to Suri! :) I've granted you developer role, so you should be able to claim the ticket, now.

Actions #17

Updated by Joyce Yu 7 months ago

  • Assignee changed from OISF Dev to Joyce Yu
Actions

Also available in: Atom PDF