Actions
Bug #1272
closedSegfault in libhtp 0.5.15
Added by Andreas Herz about 10 years ago. Updated almost 10 years ago.
Affected Versions:
Effort:
Difficulty:
Label:
Description
I'm getting some segfaults with the newest suricata 2.0.3 release bundled with libhtp 0.5.15.
I can't trigger it to reproduce it directly, but running suricata on one of our productive systems the segfault triggers from time to time, so it seems to be some special http traffic that is troublesome for libhtp.
Coredump:
Thread 7 (LWP 29696):
#0 0xb7498b7e in recv () from /lib/libpthread.so.0
#1 0x0815b3d5 in recv (__flags=0, __n=<optimized out>, __buf=<optimized out>, __fd=<optimized out>) at /usr/include/bits/socket2.h:45
#2 NFQRecvPkt (t=0x82465c0 <nfq_q>, tv=0x8246480 <nfq_t>) at source-nfq.c:862
#3 0x0815b8b5 in ReceiveNFQLoop (tv=0xafe6560, data=0x8246480 <nfq_t>, slot=0xabeb458) at source-nfq.c:990
#4 0x081838da in TmThreadsSlotPktAcqLoop (td=0xafe6560) at tm-threads.c:703
#5 0xb749199b in ?? () from /lib/libpthread.so.0
#6 0xb73f547e in clone () from /lib/libc.so.6
Thread 6 (LWP 29699):
#0 0xb7495a02 in pthread_cond_wait () from /lib/libpthread.so.0
#1 0x0817e09f in TmqhInputFlow (tv=0xaa780b0) at tmqh-flow.c:93
#2 0x08183ffa in TmThreadsSlotVar (td=0xaa780b0) at tm-threads.c:810
#3 0xb749199b in ?? () from /lib/libpthread.so.0
#4 0xb73f547e in clone () from /lib/libc.so.6
Thread 5 (LWP 29689):
#0 0xb73befdc in nanosleep () from /lib/libc.so.6
#1 0xb73ef04c in usleep () from /lib/libc.so.6
#2 0x08179324 in main (argc=8, argv=0xbf83e2d4) at suricata.c:2357
Thread 4 (LWP 29700):
#0 0xb7495a02 in pthread_cond_wait () from /lib/libpthread.so.0
#1 0x0817f89f in TmqhInputSimple (t=0xaab0a78) at tmqh-simple.c:55
#2 0x08183ffa in TmThreadsSlotVar (td=0xaab0a78) at tm-threads.c:810
#3 0xb749199b in ?? () from /lib/libpthread.so.0
#4 0xb73f547e in clone () from /lib/libc.so.6
Thread 3 (LWP 29698):
#0 0xb7495a02 in pthread_cond_wait () from /lib/libpthread.so.0
#1 0x0817e09f in TmqhInputFlow (tv=0xb48bd40) at tmqh-flow.c:93
#2 0x08183ffa in TmThreadsSlotVar (td=0xb48bd40) at tm-threads.c:810
#3 0xb749199b in ?? () from /lib/libpthread.so.0
#4 0xb73f547e in clone () from /lib/libc.so.6
Thread 2 (LWP 29701):
#0 0xb7495d95 in pthread_cond_timedwait () from /lib/libpthread.so.0
#1 0x08121161 in FlowManagerThread (td=0xba7d1d0) at flow-manager.c:545
#2 0xb749199b in ?? () from /lib/libpthread.so.0
#3 0xb73f547e in clone () from /lib/libc.so.6
Thread 1 (LWP 29697):
#0 0xb77709f7 in htp_tx_res_process_body_data_ex () from /usr/lib/libhtp-0.5.15.so.1
#1 0xb7771f7b in htp_tx_state_response_complete_ex () from /usr/lib/libhtp-0.5.15.so.1
#2 0xb776e3fc in htp_connp_RES_FINALIZE () from /usr/lib/libhtp-0.5.15.so.1
#3 0xb776df4a in htp_connp_res_data () from /usr/lib/libhtp-0.5.15.so.1
#4 0xb776881a in htp_connp_close () from /usr/lib/libhtp-0.5.15.so.1
#5 0x0806adf6 in HTPHandleResponseData (f=0x8e2d4a8, htp_state=0xb2b5f840, pstate=0xb2b5f818,
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-a"..., input_len=795, local_data=0x0) at app-layer-htp.c:820
#6 0x0806feba in AppLayerParserParse (alp_tctx=0xb5900a48, f=0x8e2d4a8, alproto=1, flags=11 '\v',
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-a"..., input_len=795) at app-layer-parser.c:836
#7 0x08056590 in AppLayerHandleTCPData (tv=0xaaa46c8, ra_ctx=0xb5900880, p=0x8d4e150, f=0x8e2d4a8, ssn=0xb2b5f788, stream=0xb2b5f790,
data=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-a"..., data_len=795, flags=11 '\v') at app-layer.c:288
#8 0x0816f2e5 in StreamTcpReassembleInlineAppLayer (tv=0xaaa46c8, ra_ctx=0xb5900880, ssn=0xb2b5f788, stream=0xb2b5f790, p=0x8d4e150)
at stream-tcp-reassemble.c:2361
#9 0x08173c24 in StreamTcpReassembleHandleSegment (tv=0xaaa46c8, ra_ctx=0xb5900880, ssn=0xb2b5f788, stream=0xb2b5f790, p=0x8d4e150, pq=0xb5900474)
at stream-tcp-reassemble.c:3548
#10 0x0816af85 in StreamTcpPacketStateFinWait1 (tv=0xaaa46c8, p=0x8d4e150, stt=0xb5900468, ssn=0xb2b5f788, pq=0xb5900474) at stream-tcp.c:2917
#11 0x0816c500 in StreamTcpPacket (tv=0xaaa46c8, p=0x8d4e150, stt=0xb5900468, pq=0xb49d970) at stream-tcp.c:4344
#12 0x0816d83c in StreamTcp (tv=0xaaa46c8, p=0x8d4e150, data=0xb5900468, pq=0xb49d970, postpq=0xb49d9c4) at stream-tcp.c:4581
#13 0x08183d0e in TmThreadsSlotVarRun (tv=0xaaa46c8, p=0x8d4e150, slot=0xb49d950) at tm-threads.c:559
#14 0x08184016 in TmThreadsSlotVar (td=0xaaa46c8) at tm-threads.c:814
#15 0xb749199b in ?? () from /lib/libpthread.so.0
#16 0xb73f547e in clone () from /lib/libc.so.6
Suricata:
This is Suricata version 2.0.3 RELEASE Features: NFQ PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG HAVE_HTP_URI_NORMALIZE_HOOK HAVE_NSS SIMD support: none Atomic intrisics: 1 2 4 8 byte(s) 32-bits, Little-endian architecture GCC version 4.4.6 20110731 (Red Hat 4.4.6-3), C version 199901 compiled with -fstack-protector compiled with _FORTIFY_SOURCE=2 L1 cache line size (CLS)=64 compiled with LibHTP v0.5.15, linked against LibHTP v0.5.15 Suricata Configuration: AF_PACKET support: yes PF_RING support: no NFQueue support: yes NFLOG support: no IPFW support: no DAG enabled: no Napatech enabled: no Unix socket enabled: no Detection enabled: yes libnss support: yes libnspr support: yes libjansson support: no Prelude support: no PCRE jit: no LUA support: no libluajit: no libgeoip: no Non-bundled htp: no Old barnyard2 support: no CUDA enabled: no Suricatasc install: no Unit tests enabled: no Debug output enabled: no Debug validation enabled: no Profiling enabled: no Profiling locks enabled: no Coccinelle / spatch: no Generic build parameters: Installation prefix (--prefix): /usr Configuration directory (--sysconfdir): /etc/suricata/ Log directory (--localstatedir) : /var/log/suricata/ Host: i686-redhat-linux-gnu GCC binary: gcc -std=gnu99 GCC Protect enabled: yes GCC march native enabled: no GCC Profile enabled: no
Files
Updated by Andreas Herz about 10 years ago
Suricata is started with these parameters:
/usr/sbin/suricata -c /etc/suricata/suricata.conf.inline -q 1 -vv --pidfile /var/run/suricata-inline.pid
And the config:
Updated by Andreas Herz about 10 years ago
With "set print elements 0" the relevant part looks like this:
#5 0x0806adf6 in HTPHandleResponseData (f=0x8e2d4a8, htp_state=0xb2b5f840, pstate=0xb2b5f818,
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 11 Sep 2014 11:33:16 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 10 Sep 2014 11:33:16 GMT\r\nX-Varnish: 2522529483\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish3\r\n\r\n\037\213\b", input_len=795, local_data=0x0) at app-layer-htp.c:820
#6 0x0806feba in AppLayerParserParse (alp_tctx=0xb5900a48, f=0x8e2d4a8, alproto=1, flags=11 '\v',
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 11 Sep 2014 11:33:16 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 10 Sep 2014 11:33:16 GMT\r\nX-Varnish: 2522529483\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish3\r\n\r\n\037\213\b", input_len=795) at app-layer-parser.c:836
#7 0x08056590 in AppLayerHandleTCPData (tv=0xaaa46c8, ra_ctx=0xb5900880, p=0x8d4e150, f=0x8e2d4a8, ssn=0xb2b5f788, stream=0xb2b5f790,
data=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 11 Sep 2014 11:33:16 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 10 Sep 2014 11:33:16 GMT\r\nX-Varnish: 2522529483\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish3\r\n\r\n\037\213\b", data_len=795, flags=11 '\v') at app-layer.c:288
Can't see anything special within this HTTP dump :/
Updated by Andreas Herz about 10 years ago
And i got a new one:
#5 0x0806adf6 in HTPHandleResponseData (f=0xb01fc118, htp_state=0xb1e2d400, pstate=0xb1e89fb8,
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www209\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 18 Sep 2014 05:56:44 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 17 Sep 2014 05:56:45 GMT\r\nX-Varnish: 2588944036\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish7\r\n\r\n\037\213\b", input_len=1400, local_data=0x0) at app-layer-htp.c:820
#6 0x0806feba in AppLayerParserParse (alp_tctx=0xb5900a48, f=0xb01fc118, alproto=1, flags=11 '\v',
input=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www209\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 18 Sep 2014 05:56:44 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 17 Sep 2014 05:56:45 GMT\r\nX-Varnish: 2588944036\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish7\r\n\r\n\037\213\b", input_len=1400) at app-layer-parser.c:836
#7 0x08056590 in AppLayerHandleTCPData (tv=0xaaa46c8, ra_ctx=0xb5900880, p=0x8d1f530, f=0xb01fc118, ssn=0xaed446a8, stream=0xaed446b0,
data=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www209\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 18 Sep 2014 05:56:44 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 17 Sep 2014 05:56:45 GMT\r\nX-Varnish: 2588944036\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish7\r\n\r\n\037\213\b", data_len=1400, flags=11 '\v') at app-layer.c:288
Looks exactly the same, except for X-Web-Node and X-varnish-host, but just the number. We suspect last.fm from what you can google, the funny thing is we can't find any connect to last.fm in any logs :p And it's still strange that this triggers a segfault.
Actions
#4
Updated by Andreas Herz about 10 years ago
We did dig into the dump even more. It's definitely last.fm connection, but nothing that triggers all the time. We're still trying to reproduce it.
The data part contains a gzip which i added, when i unzip it i get the attached xml. So it could be the gzip file that could also occur the segfault. But it's still valid http traffic IMHO.
Updated by Victor Julien about 10 years ago
Are you able to record your traffic to/from last.fm so you can get a pcap?
Updated by Andreas Herz about 10 years ago
Victor Julien wrote:
Are you able to record your traffic to/from last.fm so you can get a pcap?
Sure, as soon as we find out how to trigger it. It's kinda strange, since we played around with every option in the player that is used by the client and we're sniffing now all the traffic to last.fm and just hope that we can trigger it again. For now it's not like "click on that button" and it triggers, so it might be a weird combination.
Updated by Andreas Herz about 10 years ago
We compared the both coredumps we have and it looks like a memory issue which would explain why it's not that easy to reproduce. Both dumps have similiar HTTP Header from last.fm stuff, but if you look into the data part, you can see that in the second dump there is a lot of data which couldn't relate to the HTTP part.
First dump:
#7 0x08056590 in AppLayerHandleTCPData (tv=0xaaa46c8, ra_ctx=0xb5900880, p=0x8d4e150, f=0x8e2d4a8, ssn=0xb2b5f788, stream=0xb2b5f790,
data=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 11 Sep 2014 11:33:16 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 10 Sep 2014 11:33:16 GMT\r\nX-Varnish: 2522529483\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish3\r\n\r\n\037\213\b", data_len=795, flags=11 '\v') at app-layer.c:288
288 r = AppLayerParserParse(app_tctx->alp_tctx, f, *alproto, flags, data + data_al_so_far, data_len - data_al_so_far);
First dump data part:
$1 = "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www239\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 11 Sep 2014 11:33:16 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 10 Sep 2014 11:33:16 GMT\r\nX-Varnish: 2522529483\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish3\r\n\r\n\037\213\b\000\000\000\000\000\000\003\255\224\315j\204\060\020\200\357>E\b\364Xuw\351R\212\272Pz\350\245\320C_`\252Y\rj\"fR\365\355;6\333v)+\272X\017q\030\222\357\313\317$\321\241\257+\366!Z#\265\212\371\306\017\071\023*\325\231Ty\314-\036o\357\371!\361\242\352X3\203\200\326\304\\\227\234\062\250\033\204\334\060hQ\032\214\371\223\024\354M\243P\354Y\033\241\070\303\026\322\322\345\037E\001\266A\253r\032\311\350\213h\350)RP\213\244\261\252\214\202\257\220\271t\252\255\302d\023\206Q\340B\227\266m\225\024\210\315C\020t]\347W`\320?\326\001\341\002\307\030;x\321\230H.\211r\321\326\240\326\252\276)3\262qF\254\325\351\277,\355f\033:\324\214t\312\267\337/\326-\362d\302\242I\213\225\252\037\312\214\355Ou\255\265JA\333\211#\217\376\205#\316\314\240\026\b\325%\357n\267\330{b\314\255\325\355\311\324A^\341\313\316I\213n\306d\271^au\244\245\365j``]\001\270RJ\030\062:\320\214\361\005\206W\331\263wap\033\206wk\317\024\206F\366\344\376\345\235\371\251u\357$E\364\206&\336'[\020\260vq\005\000\000\275UXQDu\"\016\021\357,6}\026\276\230\335\317[ \236\200\005\033\017_\331\237\254\257\067\242e\353wl\"th\020CP\377\254]\214Xs\317\070\370\271\347\316-\377\256\v?\033\347\006\314\362\325\fA\245\325\033>\277\212\365\241d\016\246\360i\275iv9\226\061\263I*\376\b\270~'\231PP\304\022\251+\225;\233z\001\016\265S\261\325\020\067\373\204)\310\251\247\ac\345\246Q\f\256\nPf\212\034\n\246\\;\223\316p!\"\311\336\232<\356\272\004K\027\237\255\063\346.\236\347\253\304\371\373\245\375dF\353\\E|Gl\017kK2\f\307A\224\225D\000^\243Fr\340\325\205\002\214h\352H,\200\234|\t\310\273\350\205\207o\354\202\222~\220q\\G\t\343\003\315V\307-\301\347#\231\332\251f\213\224\004B\270ln\243%Pv\224\ba\032/\343\244\026Y\375\207+4W\202<\024\b\372u\026 \266\247\233\302o\027U\210\376\001\363\317\255q\240\310\274\224tX\020\027\213A\035s\024\304k\204\034tyH\274\177\322\037j'\024\022\ns\353\177*\266\344\233\063sS\253\271r\211%\200\322+\275O\022J-sm\030\004\033\270\346\350\233P\255G\231\202\nCT\365m\254\336p\314\242\201\302K\346\310\065J\301K\342\312\222\262C\030k\214\353xt\326>z.\216\367r\340\033\260\222\270\375\235\361\005z=+\v\305\317\324\061\247\030%C\370J\243\214\t0\324\234\031\r\214z\352[\364\354\071\035\231\071\020$\205\370\345_\016M;rh\204\202nHH\023do\"\201z\375\352c\304\376:_,e\322\264\214^#\212$\023\r\000V\r\350\301^\031q\031\277\024\302\266=\201\b\016\r\276\332\034&\250\220j\302\311#v\020\235\264\267\340\036\226\275\366\330\236\356\v\037\312\025N\002\262o\226\317\347Sf\361\221\323\264\260\061\262\305E\275X\202\221\310\016\f\273]\340\f\020JY\254\366J\nK>\241\312\365\370\256\365\034BH\023\217\314\260h\244Dp\002\307/.f\261\r\354\031\313v\210\025\356\206\302)l\243\277\375r\345$\332\236\233\373`\037|\b\031\350\306\063v\276\356_\313\322\004.\356\317N\027]\017\236RW\371\035\342r\306\342\315\366\210\215\n\375\265\017\177n\211\354\024\302\331\306\274\325w\344\262\225;\345\373\004\336\036+\370\310l\357\221#\277\272\024\000\000\000\000\000\000\000\001\000\000\000\n\000\000\000 qn\267`tn\267", '\000' <repeats 40 times>, "0-S\t\314\221\266\262\327\221\266\262\314\221\266\262%K2\016L\336O\263\377\377\377\377\000\000\000\000\260#p\t", '\000' <repeats 52 times>, "Sy\214\257\002\000\000\000\254\r\000\000\334\005\000\000\350.S\t\314\221\266\262@\020\220\265\030\337O\263\236%\020\b\b-S\t\210-S\t\314\221\266\262\v", '\000' <repeats 11 times>"\204, \336O\263\036", '\000' <repeats 15 times>"\204, \336O\263\000\000\000\000\377\377\377\377\377\377\377\377%K2\016\017\215\064\267\022\000\000\000\330\336O\263\375\027\036\b\000\000\000\000>@9\267\000\000\000\000\067\001\000\000\356\377\377\377\366\005\363\t\022\000\000\000:\001\000\000A\001\000\000L\246D\267<\000\220\265<\000\220\265\020\000\220\265\000\000\000\000@\000\220\265\220\000\000\000\030\000\000\000\364/H\267P'\274\262x)\274\262\000\000\000\000>@9\267\244J[\nSy\214\257p\000\000\000\377\377\377\377\214Y\032\t@\020\220\265\200\000\220\265`Z\032\t\330\347O\263(\247D\267n\000\000\000w\000\000\000\006\062\071\267[\000\000\000<\000\000\000'", '\000' <repeats 15 times>, "H\001\000\000\000\000\000\000\004\000\000\000'\000\000\000\061\001\000\000H\000\220\265\001\000\000\000\070\001\000\000\000\000\000\000\020\000\000\000\020\000\220\265\030g\273\262\364/H\267(d\273\262\004\000\000\000\264\337O\263\216q9\267\001\000\000\000\200T\275\262\034\001\000\000 \001\000\000%\001\000\000%\001\000\000\020\000\220\265\360\002\000\000\364/H\267\060d\273\262(d\273\262\350\337O\263\232u9\267 \001\000\000\n\000\000\000Te\273\262\350\337O\263sLx\267Te\273\262\254\"\265\262\n\000\000\000\060d\273\262p\220\274\262$\001\000\000\210\350O\263)l\f\b\240\337\271\262\254\"\265\262\n\000\000\000\b\000\000\000\220\305\270\262\000\000\000\000\000\000\000\000\061\000\000\000@\020\220\265@\020\220\265l\350O\263p\340O\263\020\000\220\265\063\001\000\000\n\000\000\000\000\000\000\000\274\350O\263@\020\220\265p\220\274\262\n\000\000\b\240\337\271\262\270\350O\263\274\350O\263\020\026\273\262\360\245\275\262\n\000\000\000\000\000\000\000'\000\000\000|\346O\263\340YD\267c\004\071\267\243\245:\266\230G\001\t\340YD\267|\346O\263\n\000\000\000\020\347O\263\n\000\000\000\364/H\267\351[9\267\364/H\267H\346O\263\317\066\066\267\000\000\000\000G\332\036\b\n", '\000' <repeats 18 times>, "0", '\000' <repeats 20 times>, "\n\000\000\000\377\377\377\377\000\000\000\000\001\000\000\000\n", '\000' <repeats 23 times>, "\020\345O u", '\000' <repeats 15 times>, "P\345O\263\000\000\000\000X\365\377\377", '\000' <repeats 12 times>"\377, \377\377\377\000\000\000\000\000\000\000\000\310M6\267", '\000' <repeats 23 times>, "0", '\000' <repeats 16 times>"\240, \345O\263\n\000\000\000X\365\377\377\001\000\000\000\001\000\000\000\237\345O\263\020\000\220\265\002\000\000\000\000\000\000\000\310M6\267\237\345O\263\240\345O\263d\000\000\000G\332\036\b\020\347O\263\000\000\000\000\000\000\000\000C\332\036\b\000\000\000\000\000\000\000\000p\373O\263\250\347O\263\066\332\036\b\031\000\000\000\370\001\000\000\220\021\374\b\000\000\000\000\310F\252\n(\352O\263\240\255\023\b\b\000\000\000\020G\252\n`\357\177\265`\357\177\265", '\000' <repeats 26 times>, "\024\000\240\277\237\265\000\000\000\000\000\000\000\000\320\277\237\265", '\000' <repeats 766 times>, "679701", '\000' <repeats 52 times>"\341, \345O\263\000\000\000\000\000\000\061\067\034\347O\263\000\377\377\377\000\000 \000\335\345O\263 ", '\000' <repeats 19 times>, "\020\025*\b\000\000\000\000\004\000\000\000\000\000\000\000\025_\n\000\000\000\000\000\344\002\071\267\060\r*\b\034\347O1\364\062\064\060\063\065\067\061\310\345O\263\342\025\071\267\320\345O\263\364/H\267\214\346O\263\061\350O\263\330\346O\263\344\223@\267\214\346O\263\a\000\000\000\330\346O\263\344\002\071\267\354\345O\263\373J!\b\364/H\267T\347O\263\b\346O\263\342\025\071\267\020\346O\263\364/H\267\314\346O\263\317\243:\266\030\347O\263\344\223@\267\314\346O\263\317\243:\266\030\347O\263\344\223@\267,\346O\263\001\000\000\000T\347O\263\344\002\071\267\000\000\000\000,\346O\263\364/H\267\244\347O\263X\346O\263\321\346O\263\000\000\000\000\000\000\000\000\f\350O\263\000\377\377\377h\347 \000\315\346O\263 \347O\263\364\245:\266h\347O\263\344\223@\267\000\000\000\000\020\025*\b\000\000\000\000\004\000\000\000<\350O\263\000\377\377\377\037\372 \000H\024*\b0\r*\b\f\350O\263\350\347O\263\377,\035\b\f\350O\263\315\346O\263\004\000\000\000\377\000\000\000\000\000\000\000A\347O\263\000\000\000\000\315\346O\263\250\350O\263\004\000\000\000\f\350O\263=\347O\263 \350O\263\375\346O\263\004\000\000\000\377", '\000' <repeats 15 times>, "Sy\214\257", '\000' <repeats 12 times>, "\020U\002\t\250F\002\t\250\350O\263\230\347O\263\242\064\027\b\250\350O\263=\347O\263\004\000\000\000\377", '\000' <repeats 15 times>, "=\347O\263\270F\002\t\004\000\000\000\250\350O\263\000\000 ", '\000' <repeats 14 times>"\325, \263\215\001", '\000' <repeats 39 times>, "\b\350O\263\242\064\027\b\000\000\000\000\000\000\000\000\230\347O\263Sy\214\257\267;I\267`\357\177\265\001t\000\000\006\000\000\000\033\003\000\000`\252\232\r\270\347O\263sLx\267`\252\232\rD\346\324\b\033\003\000\000P\341\324\b\220\367\265\262@(\260\n\b\350O\263^\f\027\b\310F\252\n\200\b\220\265\220\367\265\262@(\260\nP\341\324\b\000\000\000\000\001t\000\000q\003\000\000{\002\000\000\032\263\005\224\032\263\025\224"
Second dump:
#7 0x08056590 in AppLayerHandleTCPData (tv=0xaaa46c8, ra_ctx=0xb5900880, p=0x8d1f530, f=0xb01fc118, ssn=0xaed446a8, stream=0xaed446b0,
data=0xb34fd7ec "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www209\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 18 Sep 2014 05:56:44 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 17 Sep 2014 05:56:45 GMT\r\nX-Varnish: 2588944036\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish7\r\n\r\n\037\213\b", data_len=1400, flags=11 '\v') at app-layer.c:288
288 r = AppLayerParserParse(app_tctx->alp_tctx, f, *alproto, flags, data + data_al_so_far, data_len - data_al_so_far);
Second dump data part:
$2 = "HTTP/1.0 200 OK\r\nServer: Apache/2.2.22 (Unix)\r\nX-Web-Node: www209\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Methods: POST, GET, OPTIONS\r\nAccess-Control-Max-Age: 86400\r\nCache-Control: max-age=86400\r\nExpires: Thu, 18 Sep 2014 05:56:44 GMT\r\nContent-Type: text/xml; charset=utf-8;\r\nContent-Encoding: gzip\r\nDate: Wed, 17 Sep 2014 05:56:45 GMT\r\nX-Varnish: 2588944036\r\nAge: 0\r\nVia: 1.1 varnish\r\nConnection: close\r\nX-cms: 1\r\nX-varnish-host: varnish7\r\n\r\n\037\213\b\000\000\000\000\000\000\003\355][s\333\306\025~\317\257@5\223\366\001]q\357\227VV&\261S'u\322I\353d<\315\213g\261\027\t\025\bj\b\260J\364[\372\a\372\a\372\224\067\377\261.H\335\240\b\365\221 \213\365\020\236\261MR\342\001\317\341\367\235\313\236\263\213\203\317~\232W\331?\303\262)\027\365\263=\262\217\367\262P\273\205/\353\243g{\253\066\"\275\367\331\341'\aU\234gMk\333U\363loq\262\227^i\027\247\355\322\272\223&\263\313\266l\332g{o\312\363\305^vj\217B\022\224\036\204\345w\353\307\"\311l\027\255\255\272\247\351\375\364\362\371\263=#T\222\365I6\360\347`}\205li\353\223N\344\341\372\027\017j;\017\207\177\263\253y\366\",\263\037C\331\036\314\326\257m~\354WK\333&m\016\215>\230]=\351]\343\340\264\262?\273\305\252n\017\025\321\370`v\375\374W\237\345\240J\312\205:Y\350\220*N\016f\327\317\327\277\333\027</J\177\310#6\016[\201\060)\"\342Ncd\244\243Hz\317C\201#\327!]s\375\253\327\357\276\026v\260ZV\207\307m{\372\207\331\354\354\354l\277\262M\273\037\347\263\371\252)\335\254\263\362\354\355\254\323?O\372\347\033\375\273\267l\336\334\264\313`\347\266\250B\026WU\265\266\340\263=\274w\230.y\375\263[\366\330|\205\327/nl\334]\352\246i\177\255i\264\336\030\357,bZ\n\304\245\061\250\220\214#F\215\t^\vL\265\357iz}\205\367*yS\251\331\355\017x%\246\234'PeMy\236\200\326\314mU\355]J]5\351;\n\313\177\006dO\256\304\257_\230\061\336\314\bf\314HL\366\377qzt0[\213\271\270\330M\221\363\340\313\325\034\"S\202eVvy\024 \"\t\225@\221\341\247\364=\203\345\062\214\177J\177\207eol\276\206\316\341m\213\257\177v\223\227\264\307\313\277\256\254O\224\313\276\236g\257\226\241l\356\246&\305\022\300M\216\031\207r\223p- \334\304\301\250 tD\211\237&q\223Hd\031\067\310b&-%QyO\307r\363\302\006\371\327\363\374\302\006\023=\357KOC\fI\256\331<&=a2\357EO\230\310\207\321sP\366\306\346@z\262\036=_-Nc\343\216W\315\020\061\211\000\020\223r\303\300\304$]\034~?1\231\220\326{\315QtR\"\256\204C\205I\200\065La\352\264d\311W\215%\346M\355'JN\021sK\021\223\367(\371\322\326\347\331I\372(\331Q8\nu\366\027{>\024\066\001\324dZk(5S\366\ra\246\302\304&l*d\251g\211\231\264H\351l\364\b;\037#s\314\223\340\307\062\263\063B\336\031!_\033!\277\060\302D\322_\221\324\371\372Jj\254l[\207\246\231\071\233\312\251\305\321*\314\352\305\372\255\063:\363!\332U\325\276\265U\261\232\277\335\320r\377\264\206S\367\003^\251O\274\017x\241;h\376\270W\333|\217@\342\213\036\361_\207p\264Z\236\204\207s]jp\030\066\034\024\205\307\362\370Z\247\235\245.\024\f\262\a\206\027\266\311^.*\237\276\236\354u\373\356\027w2T9AV5\210\222\340(\240\r\250p\362\062\020\216cD)\300\244(\200\205B&\362\200\242\307\266 \\X\311\344X\364$#\344\027F\310_\267\237>g\237~\361\274\063\304\316bi\312\325\266\235\253\251\276\313v\307\266\322\254|/em.\241\253\336_\337\310\251\246\331Z\330\331\031\036\352\376\341\256!\021D\240\261y\327\201\315~e\307\071\246g\207\224\060\064\260\315\341\024\000\000\000\000\000\000\000\001\000\000\000\n94 \001q\267`\004q\267\000\000\000\000\064\337O\263\020\000\000\000\004\000\000\000\001\000\000\000\n\000\000\000Ca2\016 a2\016", '\000' <repeats 24 times>"\250, b2\016L\336O\263\n\000\000\000\000\000\000\000\260#p\t\000\000\000\000\377\377\377\377\000\000\000\000 15:11 75497550avv.gem\r\n-rw-rw-r-- 1 L\214M\253\002\000\000\000\254\r\000\000\334\005\000\000\064\337O\263\036\000\000\000\024\000\000\000\000\000\000\000\001\000\000\000\n-S\t \001q\267`\004q\267", '\000' <repeats 40 times>"\342, #p\t a2\016\250b2\016 a2\016\022\000\000\000\330\336O\263\377\377\377\377\000\000\000\000>\320;\267em\r\n\000\000\000\000\000\000\000\000\366\005\363\t\022\000\000\000a\000\000\000j\000\000\000\370\377\377\377\233\325\363\ts\000\000\000\017\035\067\267\003\000\000\000\030\337O\263\375\027\036\b\224\325\363\t\t\000\000\000X\035\266\262L\214M\253s\001\000\000>\320;\267\235\034\223\nL\214M\253\\\000\000\000\377\377\377\377\214Y\032\t@\020\220\265\310\001\220\265`Z\032\t\330\347O\263(7G\267n\000\000\000w\000\000\000\006\302;\267[\000\000\000>\000\000\000\066\000\000\000\000\000\000\000>\320;\267\000\000\000\000\300\001\000\000\000\000\000\000\006\000\000\000\066\000\000\000\251\001\000\000H\000\220\265\001\000\000\000\260\001\000\000\000\000\000\000 \000\000\000\020\000\220\265\220\205C\257\364\277J\267h\202C\257\004\000\000\000\264\337O\263\216\001<\267\001\000\000\000\000\000\000\000\064\000\000\000Py\257\b\345\001\000\000\345\001\000\000\020\000\220\265(\003\000\000\364\277J\267p\202C\257h\202C\257\350\337O\263\232\005<\267\340\001\000\000\b\000\000\000W\204C\257\350\337O\263s\334z\267W\204C\257\274gB\257\b\000\000\000p\202C\257xgB\257\347\001\000\000\210\350O\263)l\f\b`\265(\260\274gB\257\b\000\000\000\b\000\000\000h\202\277\262\000\000\000\000\000\000\000\000,\001\000\000@\020\220\265@\020\220\265l\350O\263p\340O\263 1 `\n\000\000\016\000\000\000\000\000\000\000\274\350O\263@\020\220\265\250fB\257\022\000\000\n`\265(\260\270\350O\263\274\350O\263\260u#\260\200\177t\257\b\000\000\000\000\000\000\000t\001\000\000|\346O\263\340\351F\267c\224;\267E4=\266(@\001\t\340\351F\267|\346O\263\n\000\000\000\020\347O\263\n\000\000\000\364\277J\267\351\353;\267\364\277J\267H\346O\263\317\306\070\267\000\000\000\000G\332\036\b\n", '\000' <repeats 15 times>, " 0994 ", '\000' <repeats 16 times>, "\n\000\000\000\377\377\377\377 15:\001\000\000\000\n", '\000' <repeats 23 times>, "\020\345O u1 ", '\000' <repeats 12 times>, "P\345O\263\000\000\000\000X\365\377\377", '\000' <repeats 12 times>"\377, \377\377\377\000\000\000\000\000\000\000\000\310\335\070\267", '\000' <repeats 23 times>, "0", '\000' <repeats 16 times>"\240, \345O\263\n\000\000\000X\365\377\377\001\000\000\000\001\000\000\000\237\345O\263\065\066\062a\002\000\000\000\000\000\000\000\310\335\070\267\237\345O\263\240\345O\263d\000\000\000G\332\036\b\020\347O\263\000\000\000\000\000\000\000\000C\332\036\b\000\000\000\000\000\000\000\000p\373O\263\250\347O\263\066\332\036\b\031\000\000\000\065\066\063avv.gem\r\n-rw-rw-r-- 1 1994 1994 75042637 Sep 16 15:11 avvdat-7563.zip\r\n-rw-rw-r-- 1 1994 1994 3819 Sep 16 15:11 avvdat.ini\r\n-rw-rw-r-- 1 1994 1994 54 Sep 16 15:11 ceu.ini\r\n-rw-rw-r-- 1 1994 1994 26100 Sep 16 15:11 extradat.mcs\r\n-rw-rw-r-- 1 1994 1994 2314 Sep 16 15:11 gdeltaavv.ini\r\n-rw-rw-r-- 1 1994 1994 3764 Sep 16 15:11 pkgcatalog.z\r\n-rw-rw-r-- 1 1994 1994 4548 Sep 16 15:11 replica.log\r\n-rw-rw-r-- 1 1994 1994 467704 Sep 16 15:11 scmdat.pdb\r\n-rw-rw-r-- 1 1994 1994 87260 Sep 16 15:11 v2datdet.mcs\r\n-rw-rw-r-- 1 1994 1994 88244 Sep 16 15:11 v2datinstall.mcs\r\n\335\004#\243\236\203\023\a\232\031\225F$\335\251\026|\360%\275\215!q\021\026\f\006\020 \326\066d\353\036\350\303^Zdn\260\034p\372`\271\347\327!\314\304\255'\337\300V:]>\332\037r\341\000\026\065\035\327\227\264i\314[\377O\361\001\253-=\200\356\367\071\n2\265i\376\t}EaZ\231\352\"\271\225v\246\377#\256DJ\223c\237\204\254\004\331E\237\245\300\272\004\316F\256\263\324U}\342J&e\276\004\206\016\034w\365\214\266\331\323\304Xy\214\367\255\nr\245\066\065\071\071\062\060\301\360\071\361\f3V\317\274\224J1\346\003\244\345\316O\203\216[+\275,Y\324\227\201\300\034\327\276Y\324\227\201\300\034\327\276Y\324\227\201\300\034\327\276Y\324\227\201\341\345O\263\000\000\000\000\000\000B7\034\347O\263\000\377\377\377Y\324 \000\335\345O\263 \324\227\201\300\034\327\276Y\324\227\201\300\034\327\276\000\000\000\000\020\025*\b\000\000\000\000\004\000\000\000Y\324\227\201\320\021\n\000Y\324\227\201\344\222;\267\330\f\301\b\004\000\062\064\364\062\060\060\063\060\066\060\310\345O\263\342\245;\267\320\345O\263\364\277J\267\214\346O\263\061\350O\263\330\346O\263\344#C\267\214\346O\263\a\000\000\000\330\346O\263\344\222;\267\354\345O\263\373J!\b\364\277J\267T\347O\263\b\346O\263\342\245;\267\020\346O\263\364\277J\267\314\346O\263\224\063=\266\030\347O\263\344#C\267\314\346O\263\224\063=\266\030\347O\263\344#C\267,\346O\263\000\000\000\000T\347O\263\344\222;\267\000\000\000\000,\346O\263\364\277J\267\244\347O\263X\346O\263\321\346O\263\000\000\000\000\000\000\000\000\f\350O\263\000\377\377\377h\347 \000\315\346O\263 \347O\263\223\064=\266h\347O\263\344#C\267\000\000\000\000\020\025*\b\000\000\000\000\004\000\000\000<\350O\263\000\377\377\377\200\373 \000H\024*\b0\r*\b\f\350O\263\350\347O\263\377,\035\b\f\350O\263\315\346O\263\004\000\000\000\377\000\000\000\000\000\000\000A\347O\263\000\000\000\000\315\346O\263(\024*\b\004\000\000\000\f\350O\263=\347 \000 \350O\263\375\346O\263\004\000\000\000\377", '\000' <repeats 43 times>"\230, \347O\263\242\064\027\b", '\000' <repeats 12 times>, "L\214M\253\267\313K\267\000\000\000\000\337\002\000\000\005\000\000\000\220\002\000\000X \203\rH\347O\263s\334z\267X \203\rd%\323\b\220\002\000\000p \323\bp\202\277\262\300a\265\v\230\347O\263^\f\027\b\310F\252\n\200\b\220\265p\202\277\262L\214M\253\267\313K\267\000\000\000\000\b\350O\263\242\064\027\b\220\002\000\000\000\000\000\000\230\347O\263L\214M\253\267\313K\267`\357\177\265\337\002\000\000\006\000\000\000x\005\000\000\330\065\v\260\000\000\000\000x\005z\267\330\065\v\260$\372\321\bx\005\000\000\060\365\321\b\260F\324\256\210\366\260\257\b\350O\263^\f\027\b\310F\252\n\200\b\220\265\260F\324\256\210\366\260\257\060\365\321\b\000\000\000\000\337\002\000\000\035\002\000\000\035\002\000\000\067\332pN7\332\200N"
In the first dump we could recreate the foo.gz from above with stripping some data from the end but the second dump just looks like a lot of wrong data that's not related to the HTTP traffic.
Updated by Andreas Herz about 10 years ago
While i try to get suricata working with valgrind i got new coredumps. I should also mention that i still can't reproduce it, since sending the same HTTP requests don't trigger the segfault again. Also looking into the new coredumps i can exclude last.fm issues this time. It looks more and more like a memory issue, i hope i get valgrind running with suricata.
"HTTP/1.1 301 TYPO3 RealURL redirect M961\r\nDate: Tue, 23 Sep 2014 12:57:12 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nX-Powered-By: PHP/5.5.9-1ubuntu4.4\r\nSet-Cookie: fe_typo_user=d0a913de57ea1153b331a164ce5a01ef; path=/\r\nContent-Encoding: gzip\r\nVary: Accept-Encoding\r\nLocation: http://avm.de/fritz-labor/\r\nContent-Length: 0\r\nConnection: close\r\nContent-Type: text/html\r\n\r\nv\330\347\253\367\347\004\321\361\362\205X\214\t\236t\315\325nK*\a\342\063t\000\000\000\005\000\005\001\000\000\000\000\000\r\000\022\000\020\004\001\005\001\002\001\004\003\005\003\002\003\004\002\002\002\000\025\000\\", '\000' <repeats 92 times>, "^\236\265\006\377\240.2\374[\257O`\214\211\350/d\270\366\352\210\337\001\373\323K\rl\266%F8\202\273\366\004$\335g\344\243\213y<UbC(\\V\271'\214n\237kT\235\363\065v\262\230\034\035dXk\022\020\363\aV\000\302\260\215\034\357(\nCo]\211k\271\003\360\020\261T\257)1 \213T3(\206\206Y\210s\351p\224f\036b>L?b\277\000Q\f\000\031\327&\327\276\302H\252\370\304\017\272F\004:\241sv\334F\035z\t\220\250\201\367I\311Y\356\323\325\240\246Vnf\n4w\340\326\240\067\346\311\002_\374\"@?{\031V\226x\310\221\321\062M1\252\326\025\060^u\242\304\255<x\021\374\035\022\256\204\242C\323\321\313\022\030\065\365)^\373+\203I\200\373\355f,\031nkp\235H\034f\327JA]\253\026\347\373#\251\022\214\250\205jH\202\323\252\247\"\230k{\266V\276%\226\363\234\337\231.\"\225\213\300!\245\222\336S\036X|wRCb'\315\345>\304{\244\347$\022\350\315T\325_\322\030<\264E\277\a\273\266\247W\201\371\345N\370\330\v\211\006\275\276\067 (\025%\307\061\f<\370G\030i`\350l8\364\tK\210\224\204R\025\242I\215\233\061\315\006\310+\245\237\311W\215\372K\313\256\260\315@\256\aZ*\371\323>\303\373\060\274\336\006\312\350vyo\346\345#\330n\241\353\"\343\246\375\071\t\304\t\324\355T\315\225\222U\022\313\215GKz/\031\034\257\234\367\230o\200\361\230\206\005\303P\376\254R\261\251\345!\236?\265\205\311-ND\016I\023\023\231\356\235q\204\214\003\322\246\217\233\021\315\200\062\020\240r\314\263\217\\\343\336+\260\024\232\314\337\355\360\232h+\321\264\265\220\003\370\261n\264\065\r\211\302\304\060\336\216u\217j\233\377\371\354r\221~ZE\300\224\244\016|#\377\331&\335\307\351\037\253\237\bd\242\037\244E\276=\324\202\067m\274\063\327\315\337S^\347\"\242\324\375\376\322m}\t\320B\004\366\025\206\255\316u\363\200\200\222\322\372H\005}\277\267|\312\347\363\354\305\246sR\346\357C\355\006>\363\067\344\322\251d\241<n\250\305=\307\262\222\253\315\250\324-\254\271n\322s\354\306f\311\313\240\331V\344\303\037\306\373q_\310\006\246T8?\232s2\237\344\353\222\360L$#\001\343\022\311-\370\352|?\337g\214\273\060\037\273\352'V!Y\266 L\317\311Q\252K\355\274\202\367\026yX\205\066\a\003\355\301\351\201Y\212\337\301N\247\207\232\305!q\036?W\254\234\354\321\071\"\355t\032F\206\225\354\331D=\267&\003\364\035\224iq\262\026\030.\342\354\272j[\305\000C\017\304\017\274\233\037\210V7\027R\265\344\032\372\232`v\244\201\016\244rI9\303\t\271\350(\003\346\253)\237T\361\357\260~+\357X\004\352[\353ya\272\306\374\025q\026\300\215\\m\224\020F>8\336~HQ\306\340\032\234Q<o\f\032d\232#\250\363-\333\033\255&\024\253\227\\k\301\321P\340o\277v\303\353\304E\336\310\377\223VuF\350\247\241W\346\240`\235\354\276\310\003\323\354l\226\356\354\066Q\306\033Z\335\223Q\021cnn\211\v+\002K[\250\025\003\004\205-&\243U\303\304ec\331\060\235\221\234\364\004\355\004\230\036\301\352W\252\356\026N}?j\323C\357\370%\332\062\261\353\370\033\332\370\f\024\000\000\000\000\000\000\000\001\000\000\000\n\000\000\000 \201e\267`\204e\267\000\000\000\000\064\337?\263\020\000\000\000\004\000\000\000\001\000\000\000\n\000\000\000\037A2\016 A2\016", '\000' <repeats 24 times>"\326, H2\016L\336?\263\n\000\000\000\000\000\000\000\000xi\t\000\000\000\000\377\377\377\377\000\000\000\000>P0\267", '\000' <repeats 24 times>"\260, \000\220\265\000\000\000\000\000\000\000\000?\355\313_\002\000\000\000\254\r\000\000\334\005\000\000\064\337?\263\036\000\000\000\024\000\000\000\000\000\000\000\001\000\000\000\n\324h\n \201e\267`\204e\267", '\000' <repeats 40 times>, "2xi\t A2\016\326H2\016 A2\016\022\000\000\000\330\336?\263\377\377\377\377\000\000\000\000\330\002\000\000\335\002\000\000\000\000\000\000\000\000\000\000\366\005\363\t\022\000\000\000\023\001\000\000\032\001\000\000L\266;\267<\000\220\265<\000\220\265\020\000\220\265\000\000\000\000@\000\220\265H\000\000\000@\000\000\000\364??\267\370v\252\262?\355\313_1\001\000\000\346\377\377\377\235\034\223\n@\020\220\265\210\357\335\b\230#\223\n\310\347?\263H)\f\b`\036\372\b@\020\220\265\020\026\223\n\330#\223\n\210\357\335\b@\005\323\261\006B0\267\000\000\000\000\005", '\000' <repeats 11 times>, ">P0\267\000\000\000\000p\004\000\000\002\000\000\000\021\000\000\000@\020\220\265\254\337?\263\254\347?\263\001\000\000\000`\004\000\000\000\000\000\000\030\000\000\000\020\000\220\265\260\t\323\261\364??\267\070\005\323\261\004\000\000\000\264\337?\263\216\201\060\267\001\000\000\000\000\000\000\000\064\000\000\000`\247\313\b\002\000\000\000\f\000\000\000H\000\220\265x\004\000\000\020\000\000\000?\355\313_\020\000\220\265\200B\327\261\000\000\000\000\364??\267\020\000\220\265-\026`\031\350\337?\263\205y0\267\f\000\000\000\000\000\000\000DQ2\016\210B\327\261\000\000\000\000-\026`\031H\350?\263\\D\027\b\f\000\000\000\000\000\000\000\350P\325\261\b\000\000\000@\302\323\261\000\000\000\000x\350?\263\357`\b\b@\020\220\265l\340?\263l\350?\263p\340?\263\000\000\000\000'\000\000\000\n\000\000\000\000\000\000\000\274\350?\263\000\000\000\000\340\207:\000\020\034\336\b\260\364\320\b\000\307\204\n0\274\204\n`\036\372\b\000\000\000\000\350P\325\261\244\371\320\b'", '\000' <repeats 159 times>, ">P0\267", '\000' <repeats 12 times>, ">P0\267\000\000\000\000\000\000\000\000\300\000\220\265>P0\267\000\000\000\000(\267;\267n\000\000\000w\000\000\000|\000\000\000\\\000\000\000@\000\000\000@", '\000' <repeats 15 times>, "\020\002\000\000\001\000\000\000\b\000\000\000@\000\000\000\370\001\000\000H\000\220\265\000\000\002\000\000\002\000\000\000\000\000\000\020\000\220\265h\335o\265@\000\220\265\364??\267\020\000\220\265\001\000\000\000\250\341?\263\205y0\267\020\000\220\265\330\277\237\265@\000\220\265\370\001\000\000\024\000\000\000\001\000\000\000\b\352?\263\222\025\b\bp\335o\265\000\000\000\000\370\001\000\000\220\021\374\b\000\000\000\000\310F\252\n(\352?\263\240\255\023\b\b\000\000\000\020G\252\n`\357\177\265`\357\177\265", '\000' <repeats 26 times>, "\024\000\240\277\237\265\000\000\000\000\000\000\000\000\320\277\237\265", '\000' <repeats 1064 times>"\321, \346?\263\000\000\000\000\000\000\000\000\f\350?\263\000\377\377\377\000\000 \000\315\346?\263 ", '\000' <repeats 19 times>, "\020\025*\b\000\000\000\000\004\000\000\000<\350?\263\000\377\377\377\000\000 \000H\024*\b0\r*\b\f\350?\263\350\347?\263\377,\035\b\f\350?\263\315\346?\263\004\000\000\000\377\000\000\000\000\000\000\000A\347?\263\000\000\000\000\315\346?\263(\024*\b\004\000\000\000\f\350?\263=\347 \000 \350?\263\375\346?\263\004\000\000\000\377", '\000' <repeats 43 times>"\230, \347?\263\242\064\027\b", '\000' <repeats 104 times>, "\b\350?\263\242\064\027\b\000\000\000\000\000\000\000\000\230\347?\263?\355\313_\267K@\267`\357\177\265\227L\000\000\004\000\000\000k\001\000\000\000\230\226\n\270\347?\263s\\o\267\000\230\226\n\024/\321\bk\001\000\000 *\321\b\350\270\323\261 \232\342\n\b\350?\263^\f\027\b\310F\252\n\200\b\220\265\350\270\323\261 \232\342\n *\321\b\000\000\000\000\227L\000\000\340\001\000\000\340\001\000\000\301\024`\031\301\024p\031"
And:
"HTTP/1.1 301 TYPO3 RealURL redirect M961\r\nDate: Tue, 23 Sep 2014 12:46:09 GMT\r\nServer: Apache/2.4.7 (Ubuntu)\r\nX-Powered-By: PHP/5.5.9-1ubuntu4.4\r\nSet-Cookie: fe_typo_user=bd242640093e5a0da514ed8e1902aa8d; path=/\r\nContent-Encoding: gzip\r\nVary: Accept-Encoding\r\nLocation: http://avm.de/fritz-labor/\r\nContent-Length: 0\r\nConnection: close\r\nContent-Type: text/html\r\n\r\ntest-installation-avm-2599936.html\r\nCookie: fe_typo_user=619d114584bcde28aba3de691c1a35bd\r\nConnection: keep-alive\r\n\r\nrver at www.avm.de Port 80</address>\n</body></html>\n\220}\204E`\300\202\026\373e\226\032\002\362\270\377\236CR.\214\301O\340h\307\254\071\006\311\002\317:&\204\267\376\027\355\026+\224\235\207\357}\352\202\331i+\255,&zG\260'\245?=1K\250\016\004\260\357\343\070cc\366CQ\030\344@E\037c\001\337\033QrJ\202MD\344\063\362\317\337\376\225>\316\210\330\253`\316\336\301'~\242\313~\rNP\214\300f\022z\370nS\257Y\303\353\261\213\265\246\262R\005\244u\002\003\001\000\001\243\201\321\060\201\316\060\035\006\003U\035\016\004\026\004\024\374\036\307\230\035\257|\342\062\231\334\365\006\"\a\366 \021\067S0\201\241\006\003U\035#\004\201\231\060\201\226\200\024\374\036\307\230\035\257|\342\062\231\334\365\006\"\a\366 \021\067S\241{\244y0w1\v0\t\006\003U\004\006\023\002de1\021\060\017\006\003U\004\a\023\bAugsburg1\026\060\024\006\003U\004\n\023\rLinogate GmbH1\031\060\027\006\003U\004\003\023\020mail.linogate.de1\"0 \006\t*\206H\206\367\r\001\t\001\026\023support@linogate.de\202\001\000\060\t\006\003U\035\023\004\002\060\000\060\r\006\t*\206H\206\367\r\001\001\004\005\000\003\202\001\001\000C\363Bj\352EM\\\023\a\031p\362\357|\255\274v\024=\253\331g\337\f\322KE\036%\271n\025(\f\021\262z\354\237 W\351{\240\n\005\235\030\362ZC\214\300<\277j\244\323(\344\020<\227<\267w\242\337tT \320.dQ\006\nf`\003\235\353\201K\023\251T\252\264\273V\372\312 \304`\335\307\225\034\237*\242\355\256\232\016?)R\261^+\b\275\373\ni\205\216\301t\211r\315\016\"\272>\354\363\332\060\266\212\235\357\343\326\342\267\255B\354H\310\244\224\242\274\005\004\200\271\067\246\375PF\030k\270\006\036\200\300\374\065\262\357\255\276\327[\252\207\332*\331\366\323\255F\274\214'x\377\364\252\265f8t\211\270\002\070q\306\200y\261\274\322\346\360\353}\274bw\344SL\366\352\263<\267\315\335\035g\273\255k ]e\334\364Ziy\023k\357$\204\223\377\371\210\314u\215\311~{u\354q[\a\026\003\001\002\r\f\000\002\t\000\200\326}\344@\313\273\334\031\066\326\223\323J\375\n\325\f\204\322\071\244_R\v\270\201t\313\230\274\351Q\204\237\221.c\234r\373\023\264\264\327\027~\026\325Z\301y\272B\v*)\376\062JFzc^\201\377Y\001\067{\355\334\375\063\026\212F\032\255;r\332\350\206\000x\004[\a\247\333\312xt\b}\025\020\352\237\314\235\335\063\005\a\335b\333\210\256\252t}\340\364\326\342\275h\260\347\071>\017$!\216\263\000\001\002\000\200G]\005\256MZ\030\231\360H\234\062\256\225|%\314\215rTo:\304\302M>a\020\062u\273^\247\061\367>\f\306\357\353\236\315\206U\251\357\344\361\335L\225\207\312\020t\274#\026\375X\231/)/\372\031J\235\002\066A\vY\334\366\355\202\210\201\262\031AJ\310}\214\310z\343*\205\356\346\371n\362fL\226\325\327<\254\004\064\276\361\211\067\247\233\243\304\301\363WL8\326\241\036#\272\063\372%\314\063\001\000\212c\274\375\361%\357\264I(S\002K\354\024\000\000\000\000\000\000\000\001\000\000\000\n94 1g\267`4g\267\000\000\000\000\064\337?\263\020\000\000\000\004\000\000\000\001\000\000\000\n\000\000\000\037q2\016 q2\016", '\000' <repeats 24 times>, "w{2\016L\336?\263\n\000\000\000\000\000\000\000\060\353T\n\000\000\000\000\377\377\377\377\000\000\000\000 15:11 75557556avv.gem\r\n-rw-rw-r-- 1 \253\312\245\242\002\000\000\000\254\r\000\000\334\005\000\000\064\337?\263\036\000\000\000\024\000\000\000\000\000\000\000\001\000\000\000\n-S\t 1g\267`4g\267", '\000' <repeats 40 times>, "`\353T\n q2\016w{2\016\017M-\267\022\000\000\000\330\336?\263\375\027\036\b\000\000\000\000vv.gem\r\n6\001\000\000\356\377\377\377\366\005\363\t\022\000\000\000C\001\000\000@\001\000\000\370\377\377\377\233\325\363\tI\001\000\000p\325\363\t\000\316\363\t\230\347?\263;-\f\b\224\325\363\t\t\000\000\000`\276\245\262I\001\000\000\000\316\363\t\250\325\363\t\230\347?\263\242F\f\b`\276\245\262\200\225\246\256I\001\000\000\253\312\245\242\210\201T\t`\206T\t\330\347?\263\300\346\026\b\210VR\v\260\r8\256\006\362\061\267F\226\034\b\260\r8\256\204#\261\v\377\377\377\377>\000\062\267", '\000' <repeats 11 times>, "\a(&\365\257\000\000\000\000\260\327\363\t\360\305\363\t\001", '\000' <repeats 11 times>"\200, \000\000\000\020\000\220\265X`\252\257\364\357@\267\230[\252\257\004\000\000\000\264\337?\263\216\061\062\267\001\000\000\000\000\000\000\000\064\000\000\000\240o\261\b}\001\000\000}\001\000\000\020\000\220\265\300\004\000\000\364\357@\267\240[\252\257\230[\252\257\350\337?\263\232\065\062\267x\001\000\000\n\000\000\000 ]\252\257\350\337?\263s\fq\267 ]\252\257L\362\326\257\n\000\000\000\240[\252\257\b\341\t\260\200\001\000\000\210\350?\263\rE\f\b(\226\327\257L\362\326\257\n\000\000\000\004\000\000\000\240Yt\257\000\000\000\000\000\000\000\000\070\000\000\000\006\000\000\000@\020\220\265l\350?\263p\340?\263\267\373A\267\024\002\000\000\n\000\000\000\000\000\000\000\274\350?\263@\020\220\265\340\067<\267\t\000\000\004(\226\327\257\270\350?\263\274\350?\263`7\001\260\200D\226\257\n\000\000\000\000\000\000\000\340\000\000\000\230\310\275\t@\020\220\265\310\350?\263\246R\016\b\250?\311\261'\000\000\000\061\071\071\064 292540 Sep 22 15:11 75647565avv.gem\r\n-rw-rw-r-- 1 1994 1994 260548 Sep 22 15:11 75657566avv.gem\r\n-rw-rw-r-- 1 1994 1994 307300 Sep 22 15:11 75667567avv.gem\r\n-rw-rw-r-- 1 1994 1994 225044 Sep 22 15:11 75677568avv.gem\r\n-rw-rw-r-- 1 1994 1994 262076 Sep 22 15:11 75687569avv.gem\r\n-rw-rw-r-- 1 1994 1994 74247221 Sep 22 15:11 avvdat-7569.zip\r\n-rw-rw-r-- 1 1994 1994 3819 Sep 22 15:11 avvdat.ini\r\n-rw-rw-r-- 1 1994 1994 54 Sep 22 15:11 ceu.ini\r\n-rw-rw-r-- 1 1994 1994 26100 Sep 22 15:11 extradat.mcs\r\n-rw-rw-r-- 1 1994 1994 2315 Sep 22 15:11 gdeltaavv.ini\r\n-rw-rw-r-- 1 1994 1994 3772 Sep 22 15:11 pkgcatalog.z\r\n-rw-rw-r-- 1 1994 1994 4549 Sep 22 15:11 replica.log\r\n-rw-rw-r-- 1 1994 1994 467704 Sep 22 15:11 scmdat.pdb\r\n-rw-rw-r-- 1 1994 1994 87260 Sep 22 15:11 v2datdet.mcs\r\n-rw-rw-r-- 1 1994 1994 88244 Sep 22 15:11 v2datinstall.mcs\r\n2d\226\215T\037n\225\224\352I\264\366\027-\335\317\315\037\333\263]\276o\206R\332\213U\206v\227dr\307\202@\300\357\216\325\370Sw\266\035Y\313M,q\260\301f'8\035\270\257\240\300\266\340\231\025[\271\372\a\360g\341d\022\351Zu\315\225\333\335\313$\310\273\033\234\202FO\323\025\375\t\370\032\352\317E\322\355\355\212\371L\220\063A\233z\234s\323\245p\346R\346\\\254\266\225\221\333A\342H\346p\221\070px,\r%\326\251,w\346\022\vK\270\003\261\201\310\257\067\066\063\061\070\060b\324\222\315\225\346Uq\335I\353Z\032t\242\022&@V7\354~\277\375jn\311[\251nI5\315)\245-&\001\313zU-v\317\032z\314\322\021 `\n\341\345?\263\000\000\000\000\000\000\000\000\034\347?\263\000\377\377\377\\d \000\335\345?\263 \266@\340\251,s\351\364\357@\267\344\346?\263\000\000\000\000\020\025*\b\000\000\000\000\004\000\000\000\\\346?\263,\245\v\000\250\346?\263H\024*\b0\r*\b\034\347?\263\370\346?\263\377,\035\b\034\347?\263\335\345?\263\004\000\000\000\377\000\000\000\214\346?\263\061\350?\263\330\346?\263\335\345?\263(\024*\b\004\000\000\000\034\347?\263\344\302 \000\354\345?\263\373J!\b\364\357@\267T", '\000' <repeats 87 times>"\321, \346?\263\000\000\000\000\000\000\000\000\f\350?\263\000\377\377\377\000\000 \000\315\346?\263 ", '\000' <repeats 19 times>, "\020\025*\b\000\000\000\000\004\000\000\000<\350?\263\000\377\377\377\000\000 \000H\024*\b0\r*\b\f\350?\263\350\347?\263\377,\035\b\f\350?\263\315\346?\263\004\000\000\000\377\000\000\000\000\000\000\000A\347?\263\000\000\000\000\315\346?\263(\024*\b\004\000\000\000\f\350?\263=\347 \000 \350?\263\375\346?\263\004\000\000\000\377", '\000' <repeats 43 times>"\230, \347?\263\242\064\027\b", '\000' <repeats 104 times>, "\b\350?\263\242\064\027\b\000\000\000\000\000\000\000\000\230\347?\263\253\312\245\242\267\373A\267`\357\177\265(m\000\000\004\000\000\000k\001\000\000\000#\261\257\270\347?\263s\fq\267\000#\261\257\304\065\311\bk\001\000\000\320\060\311\b\250Yt\257p\256\262\257\b\350?\263^\f\027\b\310F\252\n\200\b\220\265\250Yt\257p\256\262\257\320\060\311\b\000\000\000\000(m\000\000\340Yt\257\340Yt\257\024\313{\261\024\313\213\261"
What's similar is with the new and old dumps, that it's Content-Encoding: gzip and after the Header part is the data part as you can see and in the second new dump you can also see that some totally unrelated outputs are included (like some ls or lftp output) with files and permissons etc.
I know it's hard to debug and fix a random segfault, but maybe the libhtp dev has some idea :)
Updated by Andreas Herz about 10 years ago
I tried a lot of ways to get rid of "No symbol table info available" at the libhtp part. I checked the whole compile output, i changed libhtp/Makefile libhtp/libtool to use -O0 -ggdb instead of -O2. I'm kinda stuck with this. Can you explain what is the safe way to ensure that the libhtp part is also compiled with the necessary debug part as it is in suricata (which is fine as you can see in the dumps). I have around 10 segfault dumps but without the libhtp debug part i guess we won't find the issue for the segfault, Victor has an idea but with the debug part it would be much more helpful.
Updated by Victor Julien about 10 years ago
- Status changed from New to Assigned
- Assignee set to Victor Julien
- Target version set to 2.0.5
We've determined the chain of events:
- failure in zlib setup (Z_STREAM_ERROR) leads to tx->connp->out_decompressor == NULL and htp_connp_res_data returning HTP_STREAM_ERROR
- Suricata then still sometimes calls htp_connp_close()
- htp_connp_close() resets status' and unconditionally derefs tx->connp->out_decompressor
- as this is NULL, we get a segv
The calling of htp_connp_close may be erroneous if the htp state is in error. Talking to libhtp upstream.
Updated by Victor Julien about 10 years ago
- Status changed from Assigned to Closed
- % Done changed from 0 to 100
Fix has been merged into libhtp 0.5.x branch: https://github.com/ironbee/libhtp/pull/82
Updated by Andreas Herz about 10 years ago
I added the fix and up to now, everything was fine. But:
kernel: Detect1[26568]: segfault at 0 ip b77389f7 sp b46be4f0 error 4 in libhtp.so[b7727000+1e000]
I guess i have to enable all the debug stuff again :/
Updated by Victor Julien about 10 years ago
- Status changed from Closed to Assigned
Updated by Victor Julien almost 10 years ago
- Status changed from Assigned to Closed
Should be fixed by libhtp 0.5.16
Actions