Project

General

Profile

Actions
Copy link

Feature #6999

open
JF JH

output/json: enrich EVE w/ libmaxminddb geoip info

Feature #6999: output/json: enrich EVE w/ libmaxminddb geoip info

Added by Juliana Fajardini Reichow almost 2 years ago. Updated 3 months ago.

Status:
In Review
Priority:
Normal
Assignee:
Jerry Hardee
Target version:
9.0.0-beta1
Effort:
Difficulty:
Label:

Description

Work proposed and executed by Fandi Gunawan
PR: https://github.com/OISF/suricata/pull/10703

From PR description:
Adding optional geoip enrichment into Eve-log by setting geoip-enrichment option under eve-log configuration in suricata.yaml.
The JSON structure of geoip is based on Elastic ECS geo specification.
This is to avoid the cost of additional resources in RAM, CPU, and storage for achieving a similar result with logstash.

JF Updated by Juliana Fajardini Reichow almost 2 years ago Actions
Copy link
 #1

  • Assignee changed from Community Ticket to Fandi Gunawan

JI Updated by Jason Ish almost 2 years ago Actions
Copy link
 #2

JI Updated by Jason Ish almost 2 years ago Actions
Copy link
 #3

Some initial thoughts.

If Suricata is going to block, or alert based on the geoip keyword I think it is important to log that context along with the alert. GeoIP databases differ or get out of date so you don't want Suricata making a decision on one GeoIP database, and enriching that log with another.

What I'm not so sure about is if all events should be enriched. This is typically where I'd recommend post-processing. Moving it to Suricata because Logstash is slow isn't a very strong reason, as Logstash is generally slow and there could be other ways... Filebeat with Elastic ingest streams is probably much faster. And I'm hesitant to add yet another toggle to the config.

Thoughts? I think the ECS schema layout is good here, but should probably take a quick look at the OCSF schema for similar data as well.

FG Updated by Fandi Gunawan almost 2 years ago Actions
Copy link
 #4

Jason Ish wrote in #note-3:

Some initial thoughts.

If Suricata is going to block, or alert based on the geoip keyword I think it is important to log that context along with the alert. GeoIP databases differ or get out of date so you don't want Suricata making a decision on one GeoIP database, and enriching that log with another.

What I'm not so sure about is if all events should be enriched. This is typically where I'd recommend post-processing. Moving it to Suricata because Logstash is slow isn't a very strong reason, as Logstash is generally slow and there could be other ways... Filebeat with Elastic ingest streams is probably much faster. And I'm hesitant to add yet another toggle to the config.

Thoughts? I think the ECS schema layout is good here, but should probably take a quick look at the OCSF schema for similar data as well.

Yes you are correct, the last publicly available database is from 2018, however, MaxMind provides a more recent and updated database trough different scheme.
To enrich data, data needs to be deserialize and converted to other format is costly from my perspective. I adds geoip supports is due to Suricata supports geoip using the same library used by Logstash or Filebeat.

JF Updated by Juliana Fajardini Reichow over 1 year ago Actions
Copy link
 #5

  • Status changed from In Review to New
  • Assignee changed from Fandi Gunawan to Community Ticket

Hello there, I'm unclaiming this ticket as stale, for now. Thanks for your work so far!
If you have more time in the future and would like to come back to contribute to our projects, you'll be most welcome.

WIP PR with changes requested: https://github.com/OISF/suricata/pull/10703

VJ Updated by Victor Julien about 1 year ago Actions
Copy link
 #6

  • Target version changed from 8.0.0-beta1 to 9.0.0-beta1

JH Updated by Jerry Hardee 8 months ago Actions
Copy link
 #7

Hey all, I'd like to claim this ticket. I am adapting the original author's code with the last bit of feedback they received, updating the code for v8, plus adding enrichment to flows output.

VJ Updated by Victor Julien 8 months ago Actions
Copy link
 #8

  • Status changed from New to Assigned
  • Assignee changed from Community Ticket to Jerry Hardee

Great! I've assigned the ticket to you.

VJ Updated by Victor Julien 3 months ago Actions
Copy link
 #9

@Jerry Hardee do you have anything to share?

JH Updated by Jerry Hardee 3 months ago Actions
Copy link
 #10

I just rebased my work -- I will open a PR soon after testing it one more time

JH Updated by Jerry Hardee 3 months ago Actions
Copy link
 #11

VJ Updated by Victor Julien 3 months ago Actions
Copy link
 #13

  • Status changed from Assigned to In Review
Actions
Copy link

Also available in: PDF Atom