Project

General

Profile

Actions

Support #3807

closed

Evebox Setup

Added by Punith Raya almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Affected Versions:
Label:
Beginner

Description

Hello Team,

We have installed suricata on one server and Elasticsearch on other server, trying to push all eve.json logs to elasticsearch.
Installed evebox(rpm -Uvh https://evebox.org/files/rpm/stable/evebox-release.noarch.rpm,yum install evebox) in suricata server.

When i run the command (evebox server http://xxx.31.16.151:9200) , getting below error.

--------------------
[root@ip-172-31-46-10 ec2-user]#
2020-07-01 12:50:31 (server.go:163) <Info> -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64
2020-07-01 12:50:31 (server.go:252) <Info> -- Self test: found embedded index.html.
2020-07-01 12:50:31 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
2020-07-01 12:50:31 (configdb.go:52) <Info> -- Using in-memory configuration DB.
2020-07-01 12:50:31 (migrator.go:79) <Info> -- Updating database to version 0.
2020-07-01 12:50:31 (migrator.go:79) <Info> -- Updating database to version 1.
2020-07-01 12:50:31 (server.go:305) <Info> -- Configuring ElasticSearch datastore
2020-07-01 12:50:31 (server.go:306) <Info> -- Using ElasticSearch URL http://localhost:9200
2020-07-01 12:50:31 (server.go:308) <Info> -- Using ElasticSearch Index logstash.
2020-07-01 12:50:31 (elasticsearch.go:109) <Info> -- Event base index: logstash
2020-07-01 12:50:31 (elasticsearch.go:110) <Info> -- Event search index: logstash-*
2020-07-01 12:50:31 (server.go:335) <Error> -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dial tcp 127.0.0.1:9200: connect: connection refused
------------------
Telnet is opened between suricata server and elasticsearch server.
We have to install evebox in suricata server, in order to collect eve logs and push it to elastic search? please correct me if iam wrong.


Files

Suricata1.JPG (150 KB) Suricata1.JPG Punith Raya, 07/02/2020 05:49 AM
Eveboxweb.JPG (53.4 KB) Eveboxweb.JPG Punith Raya, 07/05/2020 02:24 PM
Rules.JPG (24.3 KB) Rules.JPG Punith Raya, 07/06/2020 02:55 AM
Actions #1

Updated by Jason Ish almost 4 years ago

Check your command line, it looks like you want:

evebox server -e http://xxx.31.16.151:9200

As for where you install EveBox, if you are using Logstash or Filebeat+Logstash to process the logs into Suricata, EveBox can go pretty much everywhere. If you want EveBox to process the log files, then yes, you have to run it where the log files are. Or you could just run the agent part where the logs are which would then forward the logs to an EveBox server running elsewhere: https://evebox.readthedocs.io/en/0.11.1/agent.html

Actions #2

Updated by Jason Ish almost 4 years ago

  • Tracker changed from Bug to Support
  • Difficulty deleted (medium)
Actions #3

Updated by Punith Raya almost 4 years ago

Hello @Jason Borden,

Thanks for the reply.
Just a question, is it possible to process the logs and visualize(as attached screenshot) with just evebox(Architecture is 1 suricata server and 1 evebox server)?

Regards,
Puneeth

Actions #4

Updated by Jason Ish almost 4 years ago

If you have Elasticsearch available, something like:

BOOKMARK_DIRECTORY=/tmp evebox server -e http://elasticsearch:9200 --input /var/log/suricata/eve.json

Replace the Elasticsearch URL with your own. If you need to connect to Evebox from a browser on a different machine, then add:
--host 0.0.0.0

If you do not have Elasticsearch, you could try something like:

BOOKMARK_DIRECTORY=/tmp evebox server --host 0.0.0.0 --datastore sqlite --input /var/log/suricata/eve.json

Actions #5

Updated by Punith Raya almost 4 years ago

Yes, i have a suricata and evebox installed on same server and other server having only elasticsearch(no logstash/filebeat).

Two queries here:
1.
When i execute : BOOKMARK_DIRECTORY=/tmp evebox server e http://xxx.31.16.xxx:9200 --input /var/log/suricata/eve.json
--------------------

2020-07-02 10:51:51 (server.go:163) <Info> -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64
2020-07-02 10:51:51 (server.go:252) <Info> -- Self test: found embedded index.html.
2020-07-02 10:51:51 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
2020-07-02 10:51:51 (configdb.go:52) <Info> -- Using in-memory configuration DB.
2020-07-02 10:51:51 (migrator.go:79) <Info> -- Updating database to version 0.
2020-07-02 10:51:51 (migrator.go:79) <Info> -- Updating database to version 1.
2020-07-02 10:51:51 (server.go:387) <Error> -- Managed PostgreSQL datastore requires a data-directory
-------------
Do i need to configure anything here to avoid this error(Managed PostgreSQL datastore requires a data-directory)

2.Do i need to add this in any specific file (--host 0.0.0.0)? if i want to connect evebox from the browser.

Actions #6

Updated by Jason Ish almost 4 years ago

Punith Raya wrote in #note-5:

Do i need to configure anything here to avoid this error(Managed PostgreSQL datastore requires a data-directory)

Did you copy the sample configuration file, evebox.yaml.example to evebox.yaml? That provides an example of using PostgreSQL, probably not what you want. The command lines above assume no configuration file, so your best off to delete it.

2.Do i need to add this in any specific file (--host 0.0.0.0)? if i want to connect evebox from the browser.

If you want to connect with a browser from a different machine yes. If just connecting from localhost, then no.

Actions #7

Updated by Punith Raya almost 4 years ago

Hello Jason,

I have configured authentication section in evebox.yaml file and below

authentication:

  1. Default: false
  2. env: EVEBOX_AUTHENTICATION_REQUIRED
    required: true
    --host 0.0.0.0
  3. Type of login required:
  4. - username -- just a username...
  5. - usernamepassword -- username and password
  6. env: EVEBOX_AUTHENTICATION_TYPE
    type: usernamepassword
    --------------------
    When is execute below command:
    [root@ip-xx-xx-xx-xx /]# BOOKMARK_DIRECTORY=/tmp evebox server --host 0.0.0.0 --datastore sqlite --input /var/log/suricata/eve.json
    2020-07-04 17:10:32 (server.go:163) <Info> -- This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64
    2020-07-04 17:10:32 (server.go:252) <Info> -- Self test: found embedded index.html.
    2020-07-04 17:10:32 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
    2020-07-04 17:10:32 (configdb.go:52) <Info> -- Using in-memory configuration DB.
    2020-07-04 17:10:32 (migrator.go:79) <Info> -- Updating database to version 0.
    2020-07-04 17:10:32 (migrator.go:79) <Info> -- Updating database to version 1.
    2020-07-04 17:10:32 (server.go:364) <Error> -- SQLite datastore requires a data-directory
    ---------------------

Added the my user by using config tool:
--------------
evebox config users D /var/lib/evebox add --username xxx
-------------

Still am not able to connect evebox from the Browser, when to access like -http://[Public IP of the server]:5636/.

Please let me know if anything am missing.

Actions #8

Updated by Jason Ish almost 4 years ago

Oops, I missed an argument in the command line I gave you:

BOOKMARK_DIRECTORY=/tmp evebox server --host 0.0.0.0 --datastore sqlite --input /var/log/suricata/eve.json -D /some/directgory

Note the -D, this is the data directory that EveBox is erroring out for you on. It can be any directory you choose. In your case you'll want it to be the same directory you used when creating the user.

Actions #9

Updated by Punith Raya almost 4 years ago

This is the output and still http://[Public IP of the server]:5636/ is not reachable.

[root@ip-xxx-xx-xx-xx evebox]# BOOKMARK_DIRECTORY=/tmp evebox server --host 0.0.0.0 --datastore sqlite --input /var/log/suricata/eve.json D /var/lib/evebox
2020-07-05 14:09:06 (server.go:163) <Info> -
This is EveBox Server version 0.11.1 (rev: 4d1b355); os=linux, arch=amd64
2020-07-05 14:09:06 (server.go:252) <Info> -- Self test: found embedded index.html.
2020-07-05 14:09:06 (geoip-service.go:44) <Warning> -- Failed to initialize geoip database: no database files found
2020-07-05 14:09:06 (configdb.go:59) <Info> -- Using configuration database file /var/lib/evebox/config.sqlite
2020-07-05 14:09:06 (sqlite.go:140) <Info> -- Configuring SQLite datastore
2020-07-05 14:09:06 (sqlite.go:146) <Info> -- SQLite event store using file /var/lib/evebox/events.sqlite
2020-07-05 14:09:06 (sqlite.go:94) <Info> -- Retention period: 0 days
2020-07-05 14:09:06 (server.go:449) <Info> -- Configuring internal eve log reader
2020-07-05 14:09:06 (server.go:131) <Info> -- Session reaper started
2020-07-05 14:09:06 (server.go:165) <Info> -- Authentication disabled.
2020-07-05 14:09:06 (server.go:261) <Info> -- Listening on [0.0.0.0]:5636
2020-07-05 14:09:06 (purger.go:45) <Info> -- Initializing event retention scheduler: period = 0 days
2020-07-05 14:09:06 (bookmarker.go:71) <Info> -- Using bookmark file /tmp/b264daf6271f51125d20d5a7715e8947.bookmark
2020-07-05 14:09:06 (bookmarker.go:159) <Info> -- Found valid bookmark, jumping to offset 10064

Actions #11

Updated by Jason Ish almost 4 years ago

I think this is now probably the firewall on the system running EveBox. You may need to open port 5636.

Actions #12

Updated by Punith Raya almost 4 years ago

Rules already added as attached screenshot.

Actions #13

Updated by Punith Raya almost 4 years ago

Issue got fixed, please ignore above comments.
Thanks for the help :)

Just last one question: Is there any limitation of event handling by evebox?

Actions #14

Updated by Jason Ish almost 4 years ago

It looks like you are using SQLite, so you'll want to set a retention period as SQLite becomes very heavy quickly. I suggest 3 days. This is done with the configuration file:

data-directory: /var/lib/evebox
database:
  type: sqlite
  retention-period: 3

input:
  enabled: true
  filename: "/var/log/suricata/eve.json" 
  bookmark-directory: /var/lib/evebox

If you have issues there, can you take it over to the EveBox Github? Thanks.

Actions #15

Updated by Jason Ish almost 4 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF