Bug #5866
closeddetect: multi-tenancy crash
Description
The config file we use:¶
%YAML 1.1
---
af-packet:
- interface: antrea-l7-tap0
threads: auto
cluster-id: 80
cluster-type: cluster_flow
defrag: no
use-mmap: yes
tpacket-v2: yes
checksum-checks: no
copy-mode: ips
copy-iface: antrea-l7-tap1
- interface: antrea-l7-tap1
threads: auto
cluster-id: 81
cluster-type: cluster_flow
defrag: no
use-mmap: yes
tpacket-v2: yes
checksum-checks: no
copy-mode: ips
copy-iface: antrea-l7-tap0
multi-detect:
enabled: yes
selector: vlan
The extra config above which is included in /etc/suricata/suricata.yaml, and Suricata is started with command:
suricata -c /etc/suricata/suricata.yaml --af-packet
How to reproduce the issue:¶
- There is a client (assuming its IP is 10.10.0.1) and server (assuming its IP is 10.10.0.2), and the connections between the client the server are enforced to pass Suricata. Note that, the client and the server are in VLAN 1.
- Open a terminal the on client, run the command as following. The connections are expected to be passed after Suricata rules are added in subsequent steps.
for ((i=0;i<1000000;i++)) do curl http://10.10.0.2/api/v2/x; done
- Open another terminal the on client, run the command as following. The connections are expected to be rejected after Suricata rules are added in subsequent steps.
for ((i=0;i<1000000;i++)) do curl http://10.10.0.2/api/v1/x; done
- Add a tenant. Note that, DO NOT stop the command in Step 2 and 3.
- Add a config file /etc/suricata/antrea-tenant-1.yaml for the tenant as following:
%YAML 1.1 --- default-rule-path: /etc/suricata/rules rule-files: - /etc/suricata/rules/antrea-l7-networkpolicy-1.rules
- Add a rule file /etc/suricata/rules/antrea-l7-networkpolicy-1.rules for the tenant as following:
reject ip any any -> any any (msg: "Reject by AntreaClusterNetworkPolicy:ingress-allow-http-request-to-api-v2"; flow: to_server, established; sid: 1;) pass http any any -> any any (msg: "Allow http by AntreaClusterNetworkPolicy:ingress-allow-http-request-to-api-v2"; http.uri; content:"/api/v2/"; startswith; http.method; content:"GET"; sid: 2;)
- Register the tenant with the command as following:
suricatasc -c "register-tenant 1 /etc/suricata/antrea-tenant-1.yaml"
- Register the tenant handler with the command as following:
suricatasc -c "register-tenant-handler 1 vlan 1"
- Add a config file /etc/suricata/antrea-tenant-1.yaml for the tenant as following:
- After a few seconds, delete the tenant. Note that, DO NOT stop the command in Step 2 and 3.
- Unregister the tenant handler with the command as following:
suricatasc -c "unregister-tenant-handler 1 vlan 1"
- Unregister the tenant with the command as following:
suricatasc -c "register-tenant 1"
- Delete file /etc/suricata/antrea-tenant-1.yaml.
- Delete file /etc/suricata/rules/antrea-l7-networkpolicy-1.rules.
- Unregister the tenant handler with the command as following:
- Repeat Step 4 and Step 5 several times, stop at Step 4 finally, which means that the tenant is still there and corresponding rules take effect. Generally, the Suricata process will get Segment fault(coredumped) during repeating Step 4 and Step 5, or after stoping repeating for a while.
- If the Suricata process is still in good shape, stop the command in Step 2 and run it again for a while, the Suricata process might get Segment fault(coredumped) too.
Coredumped files. I got two coredumped files and open it with gdb. We can see the the proccess is broken at this line: https://github.com/OISF/suricata/blob/49713ebaa0b8edb057d60f1cfe9126946645a848/src/detect.c#L362¶
The value of det_ctx->non_pf_store_cnt should be modified unexpectedlly.
Files
Updated by Hongliang Liu almost 2 years ago
Updated by Hongliang Liu almost 2 years ago
- File deleted (
clipboard-202302160933-5zio1.png)
Updated by Hongliang Liu over 1 year ago
- Assignee changed from OISF Dev to Victor Julien
Updated by Victor Julien over 1 year ago
- Assignee changed from Victor Julien to Philippe Antoine
Updated by Philippe Antoine over 1 year ago
- Status changed from New to In Review
@Hongliang Liu does https://github.com/OISF/suricata/pull/8611 solve the issue ?
Updated by Hongliang Liu over 1 year ago
Philippe Antoine wrote in #note-5:
@Hongliang Liu does https://github.com/OISF/suricata/pull/8611 solve the issue ?
Thanks for updating the issue, I'll make a quick test according to your patch. BTW, will you backport the patch to old releases like 6.0.x we are using? Thanks.
Updated by Philippe Antoine over 1 year ago
- Label Needs backport added
Indeed, it should be backported
Updated by Philippe Antoine over 1 year ago
- Target version changed from TBD to 7.0.0-rc2
Updated by Victor Julien over 1 year ago
- Subject changed from af-packet/ips: Suricata process exits with segment fault (coredumped) to detect: multi-tenancy crash
- Label Needs backport to 6.0 added
- Label deleted (
Needs backport)
Updated by Hongliang Liu over 1 year ago
- Label Needs backport added
- Label deleted (
Needs backport to 6.0)
Thanks a lot, guys. It works perfectly with the patch! BTW, will you guys include this patch in the new 6.0.11 release? When will you release 6.0.11?
Updated by Philippe Antoine over 1 year ago
- Label Needs backport to 6.0 added
- Label deleted (
Needs backport)
First, we need to merge the fix in the master branch, then we will be able to backport it
Updated by OISF Ticketbot over 1 year ago
- Label deleted (
Needs backport to 6.0)
Updated by Philippe Antoine over 1 year ago
- Status changed from In Review to Resolved
Updated by Philippe Antoine over 1 year ago
- Status changed from Resolved to Closed