Project

General

Profile

Feature #2277

netinfo: structured information about the network. Output hierarchical network tree in events

Added by Eric Leblond 7 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:

Description

This feature will allow user to defined a network tree structure with name. During event generation, a lookup will be made to add information about the hierarchy of networks the source and destination address belong too.


Related issues

Related to Feature #2320: configure host os policy over unix socketRejected
Related to Feature #660: Update host policy from unix socketNew2012-12-05
Related to Feature #249: Configure host-os-policy from a file, like snorts host_attribute.xmlNew

History

#1 Updated by Victor Julien 7 months ago

Can you add a format suggestion?

#2 Updated by Andreas Herz 7 months ago

  • Target version set to TBD

#3 Updated by Giuseppe Longo 4 months ago

{
  "timestamp": "2018-01-04T15:44:16.018667+0100",
  "flow_id": 414428872802379,
  "in_iface": "wlp2s0",
  "event_type": "http",
  "src_ip": "192.168.1.3",
  "src_port": 45200,
  "dest_ip": "213.186.33.24",
  "dest_port": 80,
  "proto": "TCP",
  "net_info": {
    "src": [
      "Red team",
      "XS" 
    ],
    "dest": [
      "Internet",
      "XS" 
    ]
  },
  "tx_id": 0,
  "http": {
    "hostname": "www.glongo.it",
    "url": "/",
    "http_user_agent": "curl/7.52.1",
    "http_content_type": "text/html",
    "http_method": "GET",
    "protocol": "HTTP/1.1",
    "status": 200,
    "length": 17207
  }
}

Network information is added in "net_info" field

#4 Updated by Victor Julien 4 months ago

What will the config look like?

#5 Updated by Giuseppe Longo 4 months ago

The configuration consists in setting a json file that contains the information:

# Information about your networks can be defined in the file
# below using a JSON syntax and added when an event is generated.
#network-info: /etc/suricata/network.json

and network.json looks like:

[{"name": "Lecce", "addresses":["192.168.0.0/16"], "children": [{"name":"Department I", "addresses":["192.168.1.0/24", "192.168.2.0/24"], "children":[{"name":"Dev Room", "addresses":["192.168.1.3"], "children":[{"name":"DNS", "addresses":["192.168.1.254"]}]}]}]}]

#6 Updated by Victor Julien 4 months ago

Where does this json file come from? Does it have some standardized format? It seems to me that using yaml would make more sense, as this is our main config format.

#7 Updated by Peter Manev 4 months ago

Yaml format is always beneficial but in a lot of network management tools and equipment(Juniper/Cisco for example) you can export network(s) config information in JSON format already - so it would beneficial to have that possibility as well I think.

#8 Updated by Victor Julien 3 months ago

So if there are existing tools that export to JSON, there must be a standard of some sort? If there is no std is there at least compatibility to some product?

#9 Updated by Jason Ish 3 months ago

I have to agree. We should use YAML unless there is a specification out there that defines a JSON layout for specifying this stuff. Even if that spec is "cisco".

#10 Updated by Peter Manev 3 months ago

I dont think(know of) there is a unified cross vendor (maybe per vendor release, not sure) standard for exporting config data to JSON (or any other format).

#11 Updated by Giuseppe Longo 3 months ago

We would use both JSON and YAML formats. Do you agree with that?

#12 Updated by Victor Julien 3 months ago

No. I think my original question hasn't been answered yet. Where does the format come from? Is it your own invention? Is it compatible to a standard or a major product?

#13 Updated by Giuseppe Longo 3 months ago

The only one we have seen so far is device42 that is not really common and has a JSON output. Their format has far more information so it is barely non human editable. So we did choose a simpler one. We can provide a conversion script if needed.

#14 Updated by Victor Julien 3 months ago

What about netjson? http://netjson.org/docs/what.html

Ultimately, if we're inventing our own format it should be in YAML like the rest of our config. I feel the only case for using json is using an existing standard that fits well.

#15 Updated by Eric Leblond 3 months ago

netjson is a no go for me as format for Suricata. The only object we have that could match is: http://netjson.org/docs/what.html#a-simplified-networkgraph-example but it is just to define a series of nodes in a network.

What could be done with netjson is that given a topology defined in a JSON object (that means we need to define ALL routers and interface) then we could transform that in the XXX format used by Suricata.

Regarding YAML input format, I really don't like the idea because we will have an indentation hell and this will not be editable by hand easily. For instance let's take a multisite company with complex office. Their YAML definition would look like:

 - Internet:
   addresses: 0.0.0.0
   childrens:
     - WorldCompany:
       childrens:
         - China:
           childrens:
             - Beijing:
               - childrens:
                 - District 1:
                   addresses: 192.168.1.0/24
                 - Red Place:
                   addresses: 192.168.2.0/24
                 - District 2:
                   addresses: 192.168.10.0/24
             - Hong Kong:
               - childrens:
                 - Main office:
                   addresses: 192.168.11.0/24
         - Netherlands:
           childrends:
             - Amsterdam:
               - childrens:
                 - Red District:
                   addresses: 192.168.12.0/24
                 - Victor's place:
                   addresses: 192.168.13.0/24
             - Rotherdam:
               - childrens:
                 - Main office:
                   addresses: 192.168.21.0/24     

That could work but it is more complex to hand fix than a JSON object.

#16 Updated by Victor Julien 2 months ago

  • Subject changed from Output hierarchical network tree in events to netinfo: structured information about the network. Output hierarchical network tree in events

At the team meeting in Amsterdam 2018 we agreed that the format should be YAML. It's easier to hand edit, and for machine output/input yaml/json doesn't matter.

We also discussed that this should need to be integrated with the address vars somehow. Jason has some ideas on how to use 'references' in YAML. So for example 'HOME_NET' would be defined with a reference to this 'netinfo' thing.

This is also related to target based reassembly settings, and perhaps to libhtp profiles as well.

#17 Updated by Victor Julien 2 months ago

  • Related to Feature #2320: configure host os policy over unix socket added

#18 Updated by Victor Julien 2 months ago

  • Related to Feature #660: Update host policy from unix socket added

#19 Updated by Victor Julien 2 months ago

  • Related to Feature #249: Configure host-os-policy from a file, like snorts host_attribute.xml added

Also available in: Atom PDF