Project

General

Profile

Actions
Copy link

Bug #6204

open

pcre: "Conditional jump or move depends on uninitialised value(s)" in valgrind

Added by Marko Jahnke over 2 years ago. Updated about 6 hours ago.

Status:
Assigned
Priority:
Normal
Assignee:
Victor Julien
Target version:
TBD
Affected Versions:
7.0.0-rc2
Effort:
Difficulty:
Label:

Description

For suricata-7.0.0-rc2 (built on Debian Bookworm), valgrind complains in my setup as folows:


[...]
306064 Conditional jump or move depends on uninitialised value(s)
306064 at 0x1988E116: ?
306064 by 0x3A318BB7: ?
306064 Uninitialised value was created by a heap allocation
306064 at 0x48407B4: malloc (vg_replace_malloc.c:381)
306064 by 0x486116C: bstr_alloc (in /usr/lib/x86_64-linux-gnu/libhtp.so.2.0.0)
306064 by 0x2E1308: SCHTPGenerateNormalizedUri (app-layer-htp-libhtp.c:114)
306064 by 0x2DF558: HTPCallbackRequestLine (app-layer-htp.c:2342)
[...]

Please find config and valgrind logs attached.

Files

Download all files
suricata-buildinfo.txt (3.95 KB) suricata-buildinfo.txt Marko Jahnke, 07/10/2023 06:15 AM
valgrind-suricata.log (24.4 KB) valgrind-suricata.log Marko Jahnke, 07/10/2023 06:15 AM
Actions
Copy link
 #1

Updated by Victor Julien over 2 years ago

I can reproduce this, but so far only with the full ET open set.

Full stack trace:

==2192320== Invalid read of size 16
==2192320==    at 0x13E02C6D: ???
==2192320==    by 0x6F0EB67: ???
==2192320==  Address 0x6f0eb8f is 63 bytes inside a block of size 75 alloc'd
==2192320==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2192320==    by 0x5597220: bstr_alloc (bstr.c:44)
==2192320==    by 0x7F257F: SCHTPGenerateNormalizedUri (app-layer-htp-libhtp.c:114)
==2192320==    by 0x7D4983: HTPCallbackRequestLine (app-layer-htp.c:2349)
==2192320==    by 0x559B619: htp_hook_run_all (htp_hooks.c:127)
==2192320==    by 0x55A4D84: htp_tx_state_request_line (htp_transaction.c:1177)
==2192320==    by 0x559F026: htp_connp_REQ_LINE_complete (htp_request.c:817)
==2192320==    by 0x559F2F4: htp_connp_req_data (htp_request.c:1077)
==2192320==    by 0x7CF5E9: HTPHandleRequestData (app-layer-htp.c:900)
==2192320==    by 0x7FE5F1: AppLayerParserParse (app-layer-parser.c:1403)
==2192320==    by 0x7958AC: AppLayerHandleTCPData (app-layer.c:773)
==2192320==    by 0xB81848: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1328)
==2192320==    by 0xB81CB9: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1391)
==2192320==    by 0xB83D31: StreamTcpReassembleHandleSegmentUpdateACK (stream-tcp-reassemble.c:1949)
==2192320==    by 0xB8403A: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1997)
==2192320==    by 0xB39E98: HandleEstablishedPacketToClient (stream-tcp.c:2811)
==2192320==    by 0xB3CCFA: StreamTcpPacketStateEstablished (stream-tcp.c:3223)
==2192320==    by 0xB52D1D: StreamTcpStateDispatch (stream-tcp.c:5236)
==2192320==    by 0xB53C64: StreamTcpPacket (stream-tcp.c:5433)
==2192320==    by 0xB54EFF: StreamTcp (stream-tcp.c:5745)
==2192320==    by 0xABF7B9: FlowWorkerStreamTCPUpdate (flow-worker.c:391)
==2192320==    by 0xAC06A2: FlowWorker (flow-worker.c:607)
==2192320==    by 0x72F4E3: TmThreadsSlotVarRun (tm-threads.c:135)
==2192320==    by 0xB27D17: TmThreadsSlotProcessPkt (tm-threads.h:196)
==2192320==    by 0xB2843B: PcapFileCallbackLoop (source-pcap-file-helper.c:108)
==2192320==    by 0x556AC53: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.10.1)
==2192320==    by 0xB28709: PcapFileDispatch (source-pcap-file-helper.c:153)
==2192320==    by 0xB23B70: ReceivePcapFileLoop (source-pcap-file.c:180)
==2192320==    by 0x72FDDB: TmThreadsSlotPktAcqLoop (tm-threads.c:316)
==2192320==    by 0x569FB42: start_thread (pthread_create.c:442)
==2192320==    by 0x5730BB3: clone (clone.S:100)

Actions
Copy link
 #2

Updated by Victor Julien over 2 years ago

Not just URI related

==2192320== Invalid read of size 16
==2192320==    at 0xD9010E5: ???
==2192320==    by 0x5E38DFF: ???
==2192320==  Address 0x5e38eef is 239 bytes inside a block of size 249 alloc'd
==2192320==    at 0x48487A9: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2192320==    by 0x752588: SCReallocFunc (util-mem.c:46)
==2192320==    by 0x7F2D8E: HTPRealloc (app-layer-htp-mem.c:176)
==2192320==    by 0x7D4E00: HTPCallbackResponseHeaderData (app-layer-htp.c:2446)
==2192320==    by 0x559B619: htp_hook_run_all (htp_hooks.c:127)
==2192320==    by 0x55A16FC: htp_connp_res_receiver_send_data (htp_response.c:102)
==2192320==    by 0x55A16FC: htp_connp_res_receiver_finalize_clear (htp_response.c:120)
==2192320==    by 0x55A16FC: htp_connp_res_receiver_finalize_clear (htp_response.c:117)
==2192320==    by 0x55A4E7B: htp_tx_state_response_headers (htp_transaction.c:1365)
==2192320==    by 0x55A1FD4: htp_connp_res_data (htp_response.c:1351)
==2192320==    by 0x7CF9D9: HTPHandleResponseData (app-layer-htp.c:968)
==2192320==    by 0x7FE5F1: AppLayerParserParse (app-layer-parser.c:1403)
==2192320==    by 0x7958AC: AppLayerHandleTCPData (app-layer.c:773)
==2192320==    by 0xB81848: ReassembleUpdateAppLayer (stream-tcp-reassemble.c:1328)
==2192320==    by 0xB81CB9: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:1391)
==2192320==    by 0xB83D31: StreamTcpReassembleHandleSegmentUpdateACK (stream-tcp-reassemble.c:1949)
==2192320==    by 0xB8403A: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:1997)
==2192320==    by 0xB3899C: HandleEstablishedPacketToServer (stream-tcp.c:2666)
==2192320==    by 0xB3CC11: StreamTcpPacketStateEstablished (stream-tcp.c:3209)
==2192320==    by 0xB52D1D: StreamTcpStateDispatch (stream-tcp.c:5236)
==2192320==    by 0xB53C64: StreamTcpPacket (stream-tcp.c:5433)
==2192320==    by 0xB54EFF: StreamTcp (stream-tcp.c:5745)
==2192320==    by 0xABF7B9: FlowWorkerStreamTCPUpdate (flow-worker.c:391)
==2192320==    by 0xAC06A2: FlowWorker (flow-worker.c:607)
==2192320==    by 0x72F4E3: TmThreadsSlotVarRun (tm-threads.c:135)
==2192320==    by 0xB27D17: TmThreadsSlotProcessPkt (tm-threads.h:196)
==2192320==    by 0xB2843B: PcapFileCallbackLoop (source-pcap-file-helper.c:108)
==2192320==    by 0x556AC53: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.10.1)
==2192320==    by 0xB28709: PcapFileDispatch (source-pcap-file-helper.c:153)
==2192320==    by 0xB23B70: ReceivePcapFileLoop (source-pcap-file.c:180)
==2192320==    by 0x72FDDB: TmThreadsSlotPktAcqLoop (tm-threads.c:316)
==2192320==    by 0x569FB42: start_thread (pthread_create.c:442)
==2192320==    by 0x5730BB3: clone (clone.S:100)

Actions
Copy link
 #3

Updated by Victor Julien over 2 years ago

  • Subject changed from Valgrind memory checker warns about "Conditional jump or move depends on uninitialised value(s)" in 7.0.0-rc2 to pcre: "Conditional jump or move depends on uninitialised value(s)" in valgrind
  • Status changed from New to Assigned
  • Assignee changed from Victor Julien to Philippe Antoine
  • Target version changed from TBD to 7.0.1

This is easy to reproduce using a pcre rule

alert http any any -> any any (http.request_line; content:"GET"; pcre:"/HTTP/R"; sid:1;)

I suspect libpcre2 does some kind of prefetch, as it is always about a 16 byte read.

@Philippe Antoine can you investigate?

Actions
Copy link
 #4

Updated by Philippe Antoine over 2 years ago

I do not reproduce with
valgrind ./src/suricata -k none -r ../suricata-verify/tests/http-all-headers/input.pcap -c suricata.yaml -S pcre.rules

and pcre.rules being the one you gave as alert http any any -> any any (http.request_line; content:"GET"; pcre:"/HTTP/R"; sid:1;)

What am I missing ?

Actions
Copy link
 #5

Updated by Victor Julien over 2 years ago

Same here. But ../suricata-verify/tests/http-body-inspect/http-aptget-ids-02-s2.pcap works for me.

Actions
Copy link
 #6

Updated by Philippe Antoine over 2 years ago

Nothing with http-body-inspect/http-aptget-ids-02-s2.pcap neither

What libpcre2 and valgrind version are you running ?

I have Valgrind-3.13.0 and pcre2 10.31

Actions
Copy link
 #7

Updated by Victor Julien over 2 years ago

Some additional info on this:

https://stackoverflow.com/questions/74777619/valgrind-conditional-jump-error-with-pcre2-jit-when-reading-from-file

https://groups.google.com/g/pcre2-dev/c/5ezMLhcEu14

It seems pcre2 upstream considers it a valgrind false positive, even if the overread is really happening. It's not supposed to have bad side effects.

Actions
Copy link
 #8

Updated by Victor Julien about 2 years ago

  • Target version changed from 7.0.1 to 7.0.2
Actions
Copy link
 #9

Updated by Philippe Antoine about 2 years ago

I still cannot reproduce even if I have PCRE2 with JIT and lscpu shows that there is SSE2

Running valgrind ./src/suricata -k none -r ../suricata-verify/tests/http-body-inspect/http-aptget-ids-02-s2.pcap -c suricata.yaml -S pcre.rules

Tried with latest pcre2 10.42

@Victor Julien what are you doing different ?

Actions
Copy link
 #10

Updated by Philippe Antoine about 2 years ago

  • Target version changed from 7.0.2 to TBD
Actions
Copy link
 #11

Updated by Victor Julien about 2 years ago

  • Assignee changed from Philippe Antoine to Victor Julien
Actions
Copy link
 #12

Updated by Victor Julien about 6 hours ago

Seeing this in unittests as well.

==468535== HEAP SUMMARY:
==468535==     in use at exit: 46,394,691 bytes in 12,269 blocks
==468535==   total heap usage: 10,875,349 allocs, 10,863,080 frees, 7,088,659,199 bytes allocated
==468535== 
==468535== Searching for pointers to 12,269 not-freed blocks
==468535== Checked 90,974,504 bytes
==468535== 
==468535== LEAK SUMMARY:
==468535==    definitely lost: 0 bytes in 0 blocks
==468535==    indirectly lost: 0 bytes in 0 blocks
==468535==      possibly lost: 30,310 bytes in 17 blocks
==468535==    still reachable: 46,364,381 bytes in 12,252 blocks
==468535==         suppressed: 0 bytes in 0 blocks
==468535== Rerun with --leak-check=full to see details of leaked memory
==468535== 
==468535== Use --track-origins=yes to see where uninitialised values come from
==468535== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 0 from 0)
==468535== 
==468535== 1 errors in context 1 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x66603B4: ???
==468535== 
==468535== 
==468535== 1 errors in context 2 of 9:
==468535== Invalid read of size 16
==468535==    at 0x4869D05: ???
==468535==    by 0x66603B4: ???
==468535==  Address 0x66603cf is 47 bytes inside a block of size 56 alloc'd
==468535==    at 0x484DB80: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==468535==    by 0x2C3631: SCReallocFunc (util-mem.c:46)
==468535==    by 0x63FB96: HTPRealloc (app-layer-htp-mem.c:183)
==468535==    by 0x314FDA: HTPCallbackRequestHeaderData (app-layer-htp.c:1861)
==468535==    by 0xC79997: UnknownInlinedFun (hook.rs:101)
==468535==    by 0xC79997: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_receiver_send_data (request.rs:143)
==468535==    by 0xC77CE0: request_receiver_finalize_clear (request.rs:162)
==468535==    by 0xC77CE0: suricata_htp::connection_parser::ConnectionParser::state_request_headers (connection_parser.rs:664)
==468535==    by 0xC7E729: request_headers (request.rs:608)
==468535==    by 0xC7E729: handle_request_state (connection_parser.rs:483)
==468535==    by 0xC7E729: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_data (request.rs:1566)
==468535==    by 0xA19322: {closure#0} (connection_parser.rs:172)
==468535==    by 0xA19322: map<&mut suricata_htp::connection_parser::ConnectionParser, suricata_htp::connection_parser::HtpStreamState, suricata_htp::c_api::connection_parser::htp_connp_request_data::{closure_env#0}> (option.rs:1158)
==468535==    by 0xA19322: htp_connp_request_data (connection_parser.rs:171)
==468535==    by 0x312E16: HTPHandleRequestData (app-layer-htp.c:814)
==468535==    by 0x32F7A5: AppLayerParserParse (app-layer-parser.c:1387)
==468535==    by 0x481356: DetectEngineHttpRawHeaderTest27 (detect-http-raw-header.c:1909)
==468535==    by 0x30159C: UtRunTests (util-unittest.c:212)
==468535== 
==468535== 
==468535== 1 errors in context 3 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x65DDDB4: ???
==468535== 
==468535== 
==468535== 1 errors in context 4 of 9:
==468535== Invalid read of size 16
==468535==    at 0x4869D05: ???
==468535==    by 0x65DDDB4: ???
==468535==  Address 0x65dddcf is 47 bytes inside a block of size 56 alloc'd
==468535==    at 0x484DB80: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==468535==    by 0x2C3631: SCReallocFunc (util-mem.c:46)
==468535==    by 0x63FB96: HTPRealloc (app-layer-htp-mem.c:183)
==468535==    by 0x314FDA: HTPCallbackRequestHeaderData (app-layer-htp.c:1861)
==468535==    by 0xC79997: UnknownInlinedFun (hook.rs:101)
==468535==    by 0xC79997: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_receiver_send_data (request.rs:143)
==468535==    by 0xC77CE0: request_receiver_finalize_clear (request.rs:162)
==468535==    by 0xC77CE0: suricata_htp::connection_parser::ConnectionParser::state_request_headers (connection_parser.rs:664)
==468535==    by 0xC7E729: request_headers (request.rs:608)
==468535==    by 0xC7E729: handle_request_state (connection_parser.rs:483)
==468535==    by 0xC7E729: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_data (request.rs:1566)
==468535==    by 0xA19322: {closure#0} (connection_parser.rs:172)
==468535==    by 0xA19322: map<&mut suricata_htp::connection_parser::ConnectionParser, suricata_htp::connection_parser::HtpStreamState, suricata_htp::c_api::connection_parser::htp_connp_request_data::{closure_env#0}> (option.rs:1158)
==468535==    by 0xA19322: htp_connp_request_data (connection_parser.rs:171)
==468535==    by 0x312E16: HTPHandleRequestData (app-layer-htp.c:814)
==468535==    by 0x32F7A5: AppLayerParserParse (app-layer-parser.c:1387)
==468535==    by 0x48025A: DetectEngineHttpRawHeaderTest25 (detect-http-raw-header.c:1745)
==468535==    by 0x30159C: UtRunTests (util-unittest.c:212)
==468535== 
==468535== 
==468535== 1 errors in context 5 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x61B05A4: ???
==468535== 
==468535== 
==468535== 1 errors in context 6 of 9:
==468535== Invalid read of size 16
==468535==    at 0x4869D05: ???
==468535==    by 0x61B05A4: ???
==468535==  Address 0x61b05bf is 47 bytes inside a block of size 56 alloc'd
==468535==    at 0x484DB80: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==468535==    by 0x2C3631: SCReallocFunc (util-mem.c:46)
==468535==    by 0x63FB96: HTPRealloc (app-layer-htp-mem.c:183)
==468535==    by 0x314FDA: HTPCallbackRequestHeaderData (app-layer-htp.c:1861)
==468535==    by 0xC79997: UnknownInlinedFun (hook.rs:101)
==468535==    by 0xC79997: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_receiver_send_data (request.rs:143)
==468535==    by 0xC77CE0: request_receiver_finalize_clear (request.rs:162)
==468535==    by 0xC77CE0: suricata_htp::connection_parser::ConnectionParser::state_request_headers (connection_parser.rs:664)
==468535==    by 0xC7E729: request_headers (request.rs:608)
==468535==    by 0xC7E729: handle_request_state (connection_parser.rs:483)
==468535==    by 0xC7E729: suricata_htp::request::<impl suricata_htp::connection_parser::ConnectionParser>::request_data (request.rs:1566)
==468535==    by 0xA19322: {closure#0} (connection_parser.rs:172)
==468535==    by 0xA19322: map<&mut suricata_htp::connection_parser::ConnectionParser, suricata_htp::connection_parser::HtpStreamState, suricata_htp::c_api::connection_parser::htp_connp_request_data::{closure_env#0}> (option.rs:1158)
==468535==    by 0xA19322: htp_connp_request_data (connection_parser.rs:171)
==468535==    by 0x312E16: HTPHandleRequestData (app-layer-htp.c:814)
==468535==    by 0x32F7A5: AppLayerParserParse (app-layer-parser.c:1387)
==468535==    by 0x47E8E0: DetectEngineHttpRawHeaderTest22 (detect-http-raw-header.c:1497)
==468535==    by 0x30159C: UtRunTests (util-unittest.c:212)
==468535== 
==468535== 
==468535== 1 errors in context 7 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x6185004: ???
==468535== 
==468535== 
==468535== 1 errors in context 8 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x717E4B4: ???
==468535== 
==468535== 
==468535== 1 errors in context 9 of 9:
==468535== Conditional jump or move depends on uninitialised value(s)
==468535==    at 0x4869DA9: ???
==468535==    by 0x675CEC4: ???
==468535== 
==468535== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 0 from 0)

Actions
Copy link

Also available in: Atom PDF