Project

General

Profile

Actions

Bug #2518

closed

Tenant rules reload completely broken in 4.x.x

Added by Daniel Snow almost 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Hi! Bug https://redmine.openinfosecfoundation.org/issues/2090 became huger: rule-reload for multi-tenant mode is completely broken now. I've tested all versions from 4.0.0 to 4.1.0 beta - it's not working at all.

The last version where rule reload using socket commands from suricatasc works is 3.2.5.

Also, as stated in this issue https://redmine.openinfosecfoundation.org/issues/2090
, it works only when suricatasc -c reload-rules was fired first.

If someone need any logs or debug info, you're welcome to ask what specific you may need.


Related issues 2 (0 open2 closed)

Related to Suricata - Support #2517: Reload rules for tenants in Multi Tenancy modeClosedActions
Related to Suricata - Bug #2090: Rule-reload in multi-tenancy is buggyClosedVictor JulienActions
Actions #1

Updated by Daniel Snow almost 6 years ago

BTW I'm using VLAN-tenants mode, as described here
https://redmine.openinfosecfoundation.org/issues/2517

Actions #2

Updated by kai jiang almost 6 years ago

I encountered the same issue. Could you check the value of multi-detect.default in the configuration file? In my case, it's false and it could work around by setting the value to true.

Actions #3

Updated by kai jiang almost 6 years ago

BTW, I do think the unix commands for multi tenancy are broken. There are some other issues when use theses commands.

Actions #4

Updated by Andreas Herz almost 6 years ago

  • Related to Support #2517: Reload rules for tenants in Multi Tenancy mode added
Actions #5

Updated by Andreas Herz almost 6 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD
Actions #6

Updated by Andreas Herz almost 6 years ago

  • Priority changed from Urgent to Normal
Actions #7

Updated by Andreas Herz almost 6 years ago

  • Related to Bug #2090: Rule-reload in multi-tenancy is buggy added
Actions #8

Updated by Daniel Snow almost 6 years ago

kai jiang wrote:

I encountered the same issue. Could you check the value of multi-detect.default in the configuration file? In my case, it's false and it could work around by setting the value to true.

Hi Kai, thank you for sharing your experience.

Unfortunately, I've tried reloading via suricatasc with multi-detect.default set to yes,no,true,false - of course with Suricata restarting, and no one option helped to solve problem with reloading in 4.0.4.

What version you've used, in which setting the multi-detect.default helped to mitigate this bug?

Actions #9

Updated by kai jiang almost 6 years ago

There's some misunderstandings. I thought it's the crash issue described in https://redmine.openinfosecfoundation.org/issues/2090 when you ran suricatasc. Setting multi-detect.default to true can work around the segmentation fault issue.

Then what do you mean when you say suricatasc fall? Could you paste the result?

Basically, you need to set the unix-command correctly. It seems the default setting can't work:
unix-command:
enabled: auto
#filename: custom.socket

Got below errors when use the above setting:
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:137) <Error> (UnixNew) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Cannot create socket directory /usr/local/var/run/suricata/: No such file or directory
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:862) <Warning> (UnixManagerInit) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Unable to create unix command socket

You need to mkdir the directory manually or specify another file path of unix-command.filename, then run suricatasc with the specified file path.

Actions #10

Updated by Daniel Snow almost 6 years ago

kai jiang wrote:

There's some misunderstandings. I thought it's the crash issue described in https://redmine.openinfosecfoundation.org/issues/2090 when you ran suricatasc. Setting multi-detect.default to true can work around the segmentation fault issue.

Then what do you mean when you say suricatasc fall? Could you paste the result?

Basically, you need to set the unix-command correctly. It seems the default setting can't work:
unix-command:
enabled: auto
#filename: custom.socket

Got below errors when use the above setting:
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:137) <Error> (UnixNew) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Cannot create socket directory /usr/local/var/run/suricata/: No such file or directory
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:862) <Warning> (UnixManagerInit) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Unable to create unix command socket

You need to mkdir the directory manually or specify another file path of unix-command.filename, then run suricatasc with the specified file path.

Hi Kai, I have issue only with tenant reload command while using the suricatasc utility. I have unix-command.enabled=true and filename is not specified, so I assume it set as by default.

That fact that other commands like suricatasc -c 'uptime' works good states that suricatasc goes to correct Suricata unix socket path, isn't it?

Then what do you mean when you say suricatasc fall?

suricatasc just hangs on reload of tenant, it's not fail. I'm sorry for using a misleading term.

Actions #11

Updated by kai jiang almost 6 years ago

suricatasc hangs? It's much like the issue I described in https://redmine.openinfosecfoundation.org/issues/2516. Suricata runs into a dead lock in the case. You could use pstack to confirm that.

Actions #12

Updated by Victor Julien almost 6 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Victor Julien
  • Target version changed from TBD to 4.1rc1
Actions #13

Updated by Victor Julien almost 6 years ago

Can you test https://github.com/OISF/suricata/pull/3407 and report if this fixes the issue for you?

Actions #14

Updated by Daniel Snow almost 6 years ago

Victor Julien wrote:

Can you test https://github.com/OISF/suricata/pull/3407 and report if this fixes the issue for you?

Yes, I'll test this pull in a few days.

Actions #15

Updated by Victor Julien over 5 years ago

  • Status changed from Assigned to Closed
Actions #16

Updated by Daniel Snow over 5 years ago

Victor Julien wrote:

https://github.com/OISF/suricata/pull/3413

Hi Victor, I'm sorry for not being here for some weeks.

I wasn't able to test your pull requests because their branches are already merged to master and deleted.

I've build Suricata from the master branch instead and tried check if bug fixed on it:


root@ubuntu:/etc/suricata/rules# suricata -V
This is Suricata version 4.1.0-dev (rev ee576d4)
root@ubuntu:~# suricatasc -c 'reload-tenant 1 /etc/suricata/tenant-1.yaml' /var/run/suricata/suricata-command.socket
{"message": "reloading tenant succeeded", "return": "OK"}

But after some seconds Suricata that was started with command:

suricata -c /etc/suricata/suricata.yaml --af-packet

Outed this error and stopped:

suricata: util-var-name.c:297: VarNameStoreLookupById: Assertion `!(current == ((void *)0))' failed.
Aborted (core dumped)

So unfortunately seems like bug either still not fixed or while building from master I've fetched some not ready code that binded to tenants reload any way.

Don't hesitate to contact if my help will need.

Daniel.

Actions

Also available in: Atom PDF