Bug #2518
closedTenant rules reload completely broken in 4.x.x
Description
Hi! Bug https://redmine.openinfosecfoundation.org/issues/2090 became huger: rule-reload for multi-tenant mode is completely broken now. I've tested all versions from 4.0.0 to 4.1.0 beta - it's not working at all.
The last version where rule reload using socket commands from suricatasc works is 3.2.5.
Also, as stated in this issue https://redmine.openinfosecfoundation.org/issues/2090
, it works only when suricatasc -c reload-rules was fired first.
If someone need any logs or debug info, you're welcome to ask what specific you may need.
Updated by Daniel Snow over 6 years ago
BTW I'm using VLAN-tenants mode, as described here
https://redmine.openinfosecfoundation.org/issues/2517
Updated by kai jiang over 6 years ago
I encountered the same issue. Could you check the value of multi-detect.default in the configuration file? In my case, it's false and it could work around by setting the value to true.
Updated by kai jiang over 6 years ago
BTW, I do think the unix commands for multi tenancy are broken. There are some other issues when use theses commands.
Updated by Andreas Herz over 6 years ago
- Related to Support #2517: Reload rules for tenants in Multi Tenancy mode added
Updated by Andreas Herz over 6 years ago
- Assignee set to OISF Dev
- Target version set to TBD
Updated by Andreas Herz over 6 years ago
- Related to Bug #2090: Rule-reload in multi-tenancy is buggy added
Updated by Daniel Snow over 6 years ago
kai jiang wrote:
I encountered the same issue. Could you check the value of multi-detect.default in the configuration file? In my case, it's false and it could work around by setting the value to true.
Hi Kai, thank you for sharing your experience.
Unfortunately, I've tried reloading via suricatasc with multi-detect.default set to yes,no,true,false - of course with Suricata restarting, and no one option helped to solve problem with reloading in 4.0.4.
What version you've used, in which setting the multi-detect.default helped to mitigate this bug?
Updated by kai jiang over 6 years ago
There's some misunderstandings. I thought it's the crash issue described in https://redmine.openinfosecfoundation.org/issues/2090 when you ran suricatasc. Setting multi-detect.default to true can work around the segmentation fault issue.
Then what do you mean when you say suricatasc fall? Could you paste the result?
Basically, you need to set the unix-command correctly. It seems the default setting can't work:
unix-command:
enabled: auto
#filename: custom.socket
Got below errors when use the above setting:
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:137) <Error> (UnixNew) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Cannot create socket directory /usr/local/var/run/suricata/: No such file or directory
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:862) <Warning> (UnixManagerInit) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Unable to create unix command socket
You need to mkdir the directory manually or specify another file path of unix-command.filename, then run suricatasc with the specified file path.
Updated by Daniel Snow over 6 years ago
kai jiang wrote:
There's some misunderstandings. I thought it's the crash issue described in https://redmine.openinfosecfoundation.org/issues/2090 when you ran suricatasc. Setting multi-detect.default to true can work around the segmentation fault issue.
Then what do you mean when you say suricatasc fall? Could you paste the result?
Basically, you need to set the unix-command correctly. It seems the default setting can't work:
unix-command:
enabled: auto
#filename: custom.socketGot below errors when use the above setting:
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:137) <Error> (UnixNew) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Cannot create socket directory /usr/local/var/run/suricata/: No such file or directory
[15228] 21/6/2018 -- 10:07:33 - (unix-manager.c:862) <Warning> (UnixManagerInit) -- [ERRCODE: SC_ERR_INITIALIZATION(45)] - Unable to create unix command socketYou need to mkdir the directory manually or specify another file path of unix-command.filename, then run suricatasc with the specified file path.
Hi Kai, I have issue only with tenant reload command while using the suricatasc utility. I have unix-command.enabled=true and filename is not specified, so I assume it set as by default.
That fact that other commands like suricatasc -c 'uptime' works good states that suricatasc goes to correct Suricata unix socket path, isn't it?
Then what do you mean when you say suricatasc fall?
suricatasc just hangs on reload of tenant, it's not fail. I'm sorry for using a misleading term.
Updated by kai jiang over 6 years ago
suricatasc hangs? It's much like the issue I described in https://redmine.openinfosecfoundation.org/issues/2516. Suricata runs into a dead lock in the case. You could use pstack to confirm that.
Updated by Victor Julien over 6 years ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Victor Julien
- Target version changed from TBD to 4.1rc1
Updated by Victor Julien over 6 years ago
Can you test https://github.com/OISF/suricata/pull/3407 and report if this fixes the issue for you?
Updated by Daniel Snow over 6 years ago
Victor Julien wrote:
Can you test https://github.com/OISF/suricata/pull/3407 and report if this fixes the issue for you?
Yes, I'll test this pull in a few days.
Updated by Victor Julien over 6 years ago
- Status changed from Assigned to Closed
Updated by Daniel Snow over 6 years ago
Victor Julien wrote:
Hi Victor, I'm sorry for not being here for some weeks.
I wasn't able to test your pull requests because their branches are already merged to master and deleted.
I've build Suricata from the master branch instead and tried check if bug fixed on it:
root@ubuntu:/etc/suricata/rules# suricata -V
This is Suricata version 4.1.0-dev (rev ee576d4)
root@ubuntu:~# suricatasc -c 'reload-tenant 1 /etc/suricata/tenant-1.yaml' /var/run/suricata/suricata-command.socket
{"message": "reloading tenant succeeded", "return": "OK"}
But after some seconds Suricata that was started with command:
suricata -c /etc/suricata/suricata.yaml --af-packet
Outed this error and stopped:
suricata: util-var-name.c:297: VarNameStoreLookupById: Assertion `!(current == ((void *)0))' failed.
Aborted (core dumped)
So unfortunately seems like bug either still not fixed or while building from master I've fetched some not ready code that binded to tenants reload any way.
Don't hesitate to contact if my help will need.
Daniel.