Bug #3380
closedSegfault when using multi-detect
Description
Hi everyone,
We recently stumbled on a reproducible segmentation fault in Suricata, when
activating multi-tenancy.
The circumstances of this happening seems to be quite rare, but we managed to
reduce the problem and draw a perimeter around them:
- multi-tenancy must be enabled, with at least one non-null tenant
- a rule must match on PCRE
- some traffic must contain fragments to reassemble.
The segmentation fault is caused by the dereferencing of a NULL pointer,
specifically de_ctx->sig_array
which is not initialized (the whole structure is
a DETECT_ENGINE_TYPE_MT_STUB
).
The segfault happens during the analysis of a packet created by reassembling
fragmented traffic. The reason why a stub is used here might be that thetenant_id
is unset and the livedev
pointer is a null pointer too. In our test
config attached, there is only one interface configured for simplicity's sake,
and no interface in the tenant 0, but the bug happened also when there was
multiple tenants, each with different rulesets, including interfaces associated
to the default tenant (0).
(gdb) bt #0 0x00000002572f2945 in DetectPrefilterMergeSort (de_ctx=0x26d5fd7b0, det_ctx=0x3a820a59a00) at detect.c:350 #1 0x00000002572f50e5 in DetectRunPrefilterPkt (tv=0x26d603840, de_ctx=0x26d5fd7b0, det_ctx=0x3a820a59a00, p=0x3a83c99f580, scratch=0x3a8b8f1df30) at detect.c:736 #2 0x00000002572f147c in DetectRun (th_v=0x26d603840, de_ctx=0x26d5fd7b0, det_ctx=0x3a820a59a00, p=0x3a83c99f580) at detect.c:131 #3 0x00000002572fa53e in DetectFlow (tv=0x26d603840, de_ctx=0x26d5fd7b0, det_ctx=0x3a820a59a00, p=0x3a83c99f580) at detect.c:1661 #4 0x00000002572faa60 in Detect (tv=0x26d603840, p=0x3a83c99f580, data=0x3a820a59a00, pq=0x0, postpq=0x0) at detect.c:1735 #5 0x000000025743c53a in FlowWorker (tv=0x26d603840, p=0x3a83c99f580, data=0x3a83c9b6170, preq=0x26d603c20, unused=0x0) at flow-worker.c:260 #6 0x00000002575a039e in TmThreadsSlotVarRun (tv=0x26d603840, p=0x3a83c99f580, slot=0x26d603be0) at tm-threads.c:145 #7 0x00000002575a0470 in TmThreadsSlotVarRun (tv=0x26d603840, p=0x3a83c99ff20, slot=0x26d603aa0) at tm-threads.c:171 #8 0x00000002574ed291 in TmThreadsSlotProcessPkt (tv=0x26d603840, s=0x26d603aa0, p=0x3a83c99ff20) at tm-threads.h:147 #9 0x00000002574f22a5 in AFPParsePacketV3 (ptv=0x3a83c9a08c0, pbd=0x3a126400000, ppd=0x3a1264b8f18) at source-af-packet.c:1124 #10 0x00000002574f258d in AFPWalkBlock (ptv=0x3a83c9a08c0, pbd=0x3a126400000) at source-af-packet.c:1140 #11 0x00000002574f2bd0 in AFPReadFromRingV3 (ptv=0x3a83c9a08c0) at source-af-packet.c:1190 #12 0x00000002574f4d32 in ReceiveAFPLoop (tv=0x26d603840, data=0x3a83c9a08c0, slot=0x26d603960) at source-af-packet.c:1583 #13 0x00000002575a14e2 in TmThreadsSlotPktAcqLoop (td=0x26d603840) at tm-threads.c:348 #14 0x000003acae793a43 in start_thread () from /lib64/libpthread.so.0 #15 0x000003acacae990f in clone () from /lib64/libc.so.6 (gdb) p de_ctx->type $1 = DETECT_ENGINE_TYPE_MT_STUB (gdb) p de_ctx->sig_array $2 = (Signature **) 0x0 (gdb) f 1 #1 0x00000002572f50e5 in DetectRunPrefilterPkt (tv=0x26d603840, de_ctx=0x26d5fd7b0, det_ctx=0x3a820a59a00, p=0x3a83c99f580, scratch=0x3a8b8f1df30) at detect.c:736 736 in detect.c (gdb) p p->flags $3 = 37757184 (gdb) p p->tenant_id $4 = 0 (gdb) p p->livedev $5 = (struct LiveDevice_ *) 0x0
Another thing that we noticed and that we cannot fully explain is that the
first packet of the attached pcap seems to be necessary for the segfault to
occur. When removed, we could not reproduce the issue. It might be that this
packet "starts" the block and the other two are stored in the slots as "extra
packets"?
Please reach out for any additional details or tests.
Thank you.
Cheers,
Florian Maury for ArmatureTech Dev Team
Files
Updated by Gatewatcher Dev Team about 5 years ago
Just a word of context, the coredump hereabove was generated by Suricata 4.1.5. The attached config files were on Suricata 5.0 on which we also reproduced the bug.
Updated by Victor Julien about 5 years ago
- Status changed from New to Assigned
- Assignee set to Jason Ish
Updated by Gatewatcher Dev Team almost 5 years ago
Hi Jason, Victor,
Did you manage to reproduce the issue in your lab with the attached files?
This issue is a blocking one for us and our customers needing multi-tenancy. It also constitutes an easy network-triggered DoS for any Suricata user activating multi-tenancy.
At any rate, if you could provide any advice, workaround for the issue or direction regarding an acceptable solution, these would be well met.
Thank you.
Cheers,
Florian Maury for ArmatureTech Dev Team
Updated by Victor Julien almost 5 years ago
- Status changed from Assigned to Closed
- Affected Versions 5.0.1 added
- Affected Versions deleted (
4.1.5)