Actions
Bug #3639
closedvalgrind: reported leaks with pcap processing.
Affected Versions:
Effort:
Difficulty:
Label:
Description
I get the following leaks below on a pcap run with this one appearing most often
==21530== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19C5ED: PacketCopyDataOffset (decode.c:239) ==21530== by 0x19C5ED: PacketCopyData (decode.c:265) ==21530== by 0x2D8B90: PcapFileCallbackLoop (source-pcap-file-helper.c:82) ==21530== by 0x62EDDD8: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.8.1) ==21530== by 0x2D9037: PcapFileDispatch (source-pcap-file-helper.c:146) ==21530== by 0x2D5C1A: ReceivePcapFileLoop (source-pcap-file.c:176) ==21530== by 0x2F7293: TmThreadsSlotPktAcqLoop (tm-threads.c:300) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95)
fuller report on exit
valgrind --error-limit=no --suppressions=suricata/qa/valgrind.suppress --leak-check=full --track-origins=yes --log-file="valgrind.log" /opt/suricatagit/bin/suricata -k none -r mychemicalromancequarantined.pcap --set "flow.memcap=8gb" --set "stream.memcap=6gb" --set stream.reassembly.memcap=10gb --set max-pending-packets=20000 -l log/ --runmode=autofp -S "rules/*.rules" --set "app-layer.protocols.tls.ja3-fingerprints = yes" ... ... ==21530== ==21530== HEAP SUMMARY: ==21530== in use at exit: 1,031,988,655 bytes in 180,445 blocks ==21530== total heap usage: 1,102,235,167 allocs, 1,102,054,722 frees, 328,119,751,398 bytes allocated ==21530== ==21530== Thread 1 Suricata-Main: ==21530== 8 bytes in 1 blocks are possibly lost in loss record 35 of 565 ==21530== at 0x4C2FA3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x4C31D84: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x16D906: AppLayerDecoderEventsSetEventRaw (app-layer-events.c:109) ==21530== by 0x13A867: TCPProtoDetect (app-layer.c:531) ==21530== by 0x13ADF0: AppLayerHandleTCPData (app-layer.c:620) ==21530== by 0x2E8429: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1098) ==21530== by 0x2E8429: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1155) ==21530== by 0x2E9204: StreamTcpReassembleHandleSegmentUpdateACK (stream-tcp-reassemble.c:1729) ==21530== by 0x2E9204: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1772) ==21530== by 0x2DED1B: StreamTcpPacketStateFinWait1.isra.24 (stream-tcp.c:3241) ==21530== by 0x2E1C91: StreamTcpStateDispatch (stream-tcp.c:4696) ==21530== by 0x2E3D92: StreamTcpPacket (stream-tcp.c:4879) ==21530== by 0x2E4923: StreamTcp (stream-tcp.c:5215) ==21530== by 0x261C4E: FlowWorker (flow-worker.c:241) ==21530== ==21530== 8 bytes in 1 blocks are possibly lost in loss record 36 of 565 ==21530== at 0x4C2FA3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x4C31D84: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x16D906: AppLayerDecoderEventsSetEventRaw (app-layer-events.c:109) ==21530== by 0x13A867: TCPProtoDetect (app-layer.c:531) ==21530== by 0x13ADF0: AppLayerHandleTCPData (app-layer.c:620) ==21530== by 0x2E8429: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1098) ==21530== by 0x2E8429: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1155) ==21530== by 0x2E919B: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1820) ==21530== by 0x2E39BA: StreamTcpPacket (stream-tcp.c:4846) ==21530== by 0x2E4923: StreamTcp (stream-tcp.c:5215) ==21530== by 0x261C4E: FlowWorker (flow-worker.c:241) ==21530== by 0x2F6109: TmThreadsSlotVarRun (tm-threads.c:117) ==21530== by 0x2F7B31: TmThreadsSlotVar (tm-threads.c:428) ==21530== ==21530== 16 bytes in 1 blocks are possibly lost in loss record 99 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x16DA3E: AppLayerDecoderEventsSetEventRaw (app-layer-events.c:92) ==21530== by 0x13A867: TCPProtoDetect (app-layer.c:531) ==21530== by 0x13ADF0: AppLayerHandleTCPData (app-layer.c:620) ==21530== by 0x2E8429: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1098) ==21530== by 0x2E8429: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1155) ==21530== by 0x2E9204: StreamTcpReassembleHandleSegmentUpdateACK (stream-tcp-reassemble.c:1729) ==21530== by 0x2E9204: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1772) ==21530== by 0x2DED1B: StreamTcpPacketStateFinWait1.isra.24 (stream-tcp.c:3241) ==21530== by 0x2E1C91: StreamTcpStateDispatch (stream-tcp.c:4696) ==21530== by 0x2E3D92: StreamTcpPacket (stream-tcp.c:4879) ==21530== by 0x2E4923: StreamTcp (stream-tcp.c:5215) ==21530== by 0x261C4E: FlowWorker (flow-worker.c:241) ==21530== by 0x2F6109: TmThreadsSlotVarRun (tm-threads.c:117) ==21530== ==21530== 16 bytes in 1 blocks are possibly lost in loss record 100 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x16DA3E: AppLayerDecoderEventsSetEventRaw (app-layer-events.c:92) ==21530== by 0x13A867: TCPProtoDetect (app-layer.c:531) ==21530== by 0x13ADF0: AppLayerHandleTCPData (app-layer.c:620) ==21530== by 0x2E8429: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1098) ==21530== by 0x2E8429: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1155) ==21530== by 0x2E919B: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1820) ==21530== by 0x2E39BA: StreamTcpPacket (stream-tcp.c:4846) ==21530== by 0x2E4923: StreamTcp (stream-tcp.c:5215) ==21530== by 0x261C4E: FlowWorker (flow-worker.c:241) ==21530== by 0x2F6109: TmThreadsSlotVarRun (tm-threads.c:117) ==21530== by 0x2F7B31: TmThreadsSlotVar (tm-threads.c:428) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== ==21530== 104,760 bytes in 45 blocks are possibly lost in loss record 553 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260AE8: FlowForceReassemblyForFlow (flow-timeout.c:368) ==21530== by 0x260CAE: FlowForceReassemblyForHash (flow-timeout.c:452) ==21530== by 0x260CAE: FlowForceReassembly (flow-timeout.c:471) ==21530== by 0x2EA807: PostRunDeinit.part.12 (suricata.c:2032) ==21530== by 0x2F2043: PostRunDeinit (suricata.c:2022) ==21530== by 0x2F2043: SuricataMain (suricata.c:2808) ==21530== by 0x8308B96: (below main) (libc-start.c:310) ==21530== ==21530== 142,008 bytes in 61 blocks are possibly lost in loss record 554 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260ABC: FlowForceReassemblyForFlow (flow-timeout.c:361) ==21530== by 0x260CAE: FlowForceReassemblyForHash (flow-timeout.c:452) ==21530== by 0x260CAE: FlowForceReassembly (flow-timeout.c:471) ==21530== by 0x2EA807: PostRunDeinit.part.12 (suricata.c:2032) ==21530== by 0x2F2043: PostRunDeinit (suricata.c:2022) ==21530== by 0x2F2043: SuricataMain (suricata.c:2808) ==21530== by 0x8308B96: (below main) (libc-start.c:310) ==21530== ==21530== 530,784 bytes in 228 blocks are possibly lost in loss record 555 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260A3C: FlowForceReassemblyForFlow (flow-timeout.c:378) ==21530== by 0x25E7D5: FlowManagerFlowTimedOut (flow-manager.c:334) ==21530== by 0x25E7D5: FlowManagerHashRowTimeout (flow-manager.c:394) ==21530== by 0x25E7D5: FlowTimeoutHash (flow-manager.c:522) ==21530== by 0x25E7D5: FlowManager (flow-manager.c:777) ==21530== by 0x2F6E97: TmThreadsManagement (tm-threads.c:517) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 1,115,112 bytes in 479 blocks are possibly lost in loss record 556 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260AE8: FlowForceReassemblyForFlow (flow-timeout.c:368) ==21530== by 0x25E7D5: FlowManagerFlowTimedOut (flow-manager.c:334) ==21530== by 0x25E7D5: FlowManagerHashRowTimeout (flow-manager.c:394) ==21530== by 0x25E7D5: FlowTimeoutHash (flow-manager.c:522) ==21530== by 0x25E7D5: FlowManager (flow-manager.c:777) ==21530== by 0x2F6E97: TmThreadsManagement (tm-threads.c:517) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 1,122,096 bytes in 482 blocks are possibly lost in loss record 557 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260A3C: FlowForceReassemblyForFlow (flow-timeout.c:378) ==21530== by 0x260CAE: FlowForceReassemblyForHash (flow-timeout.c:452) ==21530== by 0x260CAE: FlowForceReassembly (flow-timeout.c:471) ==21530== by 0x2EA807: PostRunDeinit.part.12 (suricata.c:2032) ==21530== by 0x2F2043: PostRunDeinit (suricata.c:2022) ==21530== by 0x2F2043: SuricataMain (suricata.c:2808) ==21530== by 0x8308B96: (below main) (libc-start.c:310) ==21530== ==21530== 1,133,736 bytes in 487 blocks are possibly lost in loss record 558 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x260594: FlowForceReassemblyPseudoPacketGet (flow-timeout.c:270) ==21530== by 0x260ABC: FlowForceReassemblyForFlow (flow-timeout.c:361) ==21530== by 0x25E7D5: FlowManagerFlowTimedOut (flow-manager.c:334) ==21530== by 0x25E7D5: FlowManagerHashRowTimeout (flow-manager.c:394) ==21530== by 0x25E7D5: FlowTimeoutHash (flow-manager.c:522) ==21530== by 0x25E7D5: FlowManager (flow-manager.c:777) ==21530== by 0x2F6E97: TmThreadsManagement (tm-threads.c:517) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 1,440,600 bytes in 588 blocks are possibly lost in loss record 559 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19BF45: PacketGetFromAlloc (decode.c:146) ==21530== by 0x2F4A67: PacketPoolInit (tmqh-packetpool.c:395) ==21530== by 0x2EA7F8: PostRunDeinit.part.12 (suricata.c:2026) ==21530== by 0x2F2043: PostRunDeinit (suricata.c:2022) ==21530== by 0x2F2043: SuricataMain (suricata.c:2808) ==21530== by 0x8308B96: (below main) (libc-start.c:310) ==21530== ==21530== 2,925,300 bytes in 1,194 blocks are possibly lost in loss record 560 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19BF45: PacketGetFromAlloc (decode.c:146) ==21530== by 0x2F4A67: PacketPoolInit (tmqh-packetpool.c:395) ==21530== by 0x25D7AA: FlowManagerThreadInit (flow-manager.c:704) ==21530== by 0x2F6E66: TmThreadsManagement (tm-threads.c:504) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 27,074,640 bytes in 11,630 blocks are possibly lost in loss record 561 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x19C1F4: PacketGetFromQueueOrAlloc (decode.c:189) ==21530== by 0x2D8B05: PcapFileCallbackLoop (source-pcap-file-helper.c:64) ==21530== by 0x62EDDD8: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.8.1) ==21530== by 0x2D9037: PcapFileDispatch (source-pcap-file-helper.c:146) ==21530== by 0x2D5C1A: ReceivePcapFileLoop (source-pcap-file.c:176) ==21530== by 0x2F7293: TmThreadsSlotPktAcqLoop (tm-threads.c:300) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 28,493,500 bytes in 11,630 blocks are possibly lost in loss record 562 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19BF45: PacketGetFromAlloc (decode.c:146) ==21530== by 0x2F4A67: PacketPoolInit (tmqh-packetpool.c:395) ==21530== by 0x2F70C4: TmThreadsSlotPktAcqLoop (tm-threads.c:242) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 167,047,968 bytes in 71,756 blocks are possibly lost in loss record 563 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x35352C: SCProfilePacketStart (util-profiling.c:1229) ==21530== by 0x19BFBC: PacketGetFromAlloc (decode.c:158) ==21530== by 0x2D8B05: PcapFileCallbackLoop (source-pcap-file-helper.c:64) ==21530== by 0x62EDDD8: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.8.1) ==21530== by 0x2D9037: PcapFileDispatch (source-pcap-file-helper.c:146) ==21530== by 0x2D5C1A: ReceivePcapFileLoop (source-pcap-file.c:176) ==21530== by 0x2F7293: TmThreadsSlotPktAcqLoop (tm-threads.c:300) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 175,802,200 bytes in 71,756 blocks are possibly lost in loss record 564 of 565 ==21530== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19BF45: PacketGetFromAlloc (decode.c:146) ==21530== by 0x2D8B05: PcapFileCallbackLoop (source-pcap-file-helper.c:64) ==21530== by 0x62EDDD8: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.8.1) ==21530== by 0x2D9037: PcapFileDispatch (source-pcap-file-helper.c:146) ==21530== by 0x2D5C1A: ReceivePcapFileLoop (source-pcap-file.c:176) ==21530== by 0x2F7293: TmThreadsSlotPktAcqLoop (tm-threads.c:300) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== 625,009,308 bytes in 9,527 blocks are possibly lost in loss record 565 of 565 ==21530== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==21530== by 0x19C5ED: PacketCopyDataOffset (decode.c:239) ==21530== by 0x19C5ED: PacketCopyData (decode.c:265) ==21530== by 0x2D8B90: PcapFileCallbackLoop (source-pcap-file-helper.c:82) ==21530== by 0x62EDDD8: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.8.1) ==21530== by 0x2D9037: PcapFileDispatch (source-pcap-file-helper.c:146) ==21530== by 0x2D5C1A: ReceivePcapFileLoop (source-pcap-file.c:176) ==21530== by 0x2F7293: TmThreadsSlotPktAcqLoop (tm-threads.c:300) ==21530== by 0x693F6DA: start_thread (pthread_create.c:463) ==21530== by 0x840888E: clone (clone.S:95) ==21530== ==21530== LEAK SUMMARY: ==21530== definitely lost: 0 bytes in 0 blocks ==21530== indirectly lost: 0 bytes in 0 blocks ==21530== possibly lost: 1,031,942,060 bytes in 179,867 blocks ==21530== still reachable: 46,595 bytes in 578 blocks ==21530== suppressed: 0 bytes in 0 blocks ==21530== Reachable blocks (those to which a pointer was found) are not shown. ==21530== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==21530== ==21530== For counts of detected and suppressed errors, rerun with: -v ==21530== ERROR SUMMARY: 2266 errors from 2265 contexts (suppressed: 0 from 0)
Using
/opt/suricatagit/bin/suricata --build-info
This is Suricata version 6.0.0-dev (a95fa3c15 2020-04-10)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON PROFILING TLS MAGIC RUST
SIMD support: SSE_4_2 SSE_4_1 SSE_3
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 7.5.0, C version 199901
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.32, linked against LibHTP v0.5.32
Suricata Configuration:
AF_PACKET support: yes
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: no
Unix socket enabled: yes
Detection enabled: yes
Libmagic support: yes
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes, through luajit
libluajit: yes
GeoIP2 support: yes
Non-bundled htp: no
Old barnyard2 support:
Hyperscan support: yes
Libnet support: yes
liblz4 support: yes
Rust support: yes
Rust strict mode: no
Rust compiler path: /root/.cargo/bin/rustc
Rust compiler version: rustc 1.42.0 (b8cedc004 2020-03-09)
Cargo path: /root/.cargo/bin/cargo
Cargo version: cargo 1.42.0 (86334295e 2020-01-31)
Cargo vendor: yes
Python support: yes
Python path: /usr/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: not bundled
Profiling enabled: yes
Profiling locks enabled: no
Development settings:
Coccinelle / spatch: yes
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no
Generic build parameters:
Installation prefix: /opt/suricatagit
Configuration directory: /opt/suricatagit/etc/suricata/
Log directory: /opt/suricatagit/var/log/suricata/
--prefix /opt/suricatagit
--sysconfdir /opt/suricatagit/etc
--localstatedir /opt/suricatagit/var
--datarootdir /opt/suricatagit/share
Host: x86_64-pc-linux-gnu
Compiler: gcc (exec name) / g++ (real)
GCC Protect enabled: no
GCC march native enabled: yes
GCC Profile enabled: no
Position Independent Executable enabled: no
CFLAGS -g -O2 -march=native -I${srcdir}/../rust/gen
PCAP_CFLAGS -I/usr/include
SECCFLAGS
root@qa05:~#
Updated by Victor Julien almost 4 years ago
- Status changed from New to Feedback
- Assignee set to Peter Manev
Does this happen after a clean exit? Or did you perhaps get a message that it failed to shut certain threads down?
Updated by Peter Manev over 3 years ago
- Status changed from Feedback to Closed
Have not been able to reproduce this after latest updates for 6.0.
Closing and will reopen if i see it again.
Actions