Suricata

For some strange reason this (decrypted HTTPS) traffic is generating crashes with the most basic Lua transform. (The traffic is from a sandbox run where the malware generates lots of benign requests as part of its obfuscation of its actual intent.)
Attached PCAP.

Suricata rule is just:

alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"Test luaxform hello"; flow:established,from_server; file.data; luaxform:suri-xform-test.lua54, hello 1, world 2; content:"Hello"; sid:380000027; rev:1;)

and the `suri-xform-test.lua54` script is just:

function hex_esc(s)
    return string.gsub(s, ".", function(c) return string.format("%02x", string.byte(c)) end)
end

function transform(input, args)
    print("luaxform args: " .. table.concat(args, "|"))

    print("luaxform input size: " .. #input)
    print("luaxform input start: " .. hex_esc(input:sub(1, 128)))

    return input, #input
end

If I change the rule to remove the comma between arguments, so `hello 1 world2`, it no longer crashes.

I got a backtrace from this PCAP and rule (similar to the one in an earlier crash, before I narrowed down a rule and PCAP to trigger it):

(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimised out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimised out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimised out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x0000716a0244527e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x0000716a024288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x0000638be8c8c2c9 in luaD_throw.cold ()
#6  0x0000638be9103913 in luaH_resize ()
#7  0x0000638be91034c2 in luaH_newkey ()
#8  0x0000638be9105d5d in luaV_finishset ()
#9  0x0000638be90f44a5 in lua_settable ()
#10 0x0000638be8d38739 in TransformLuaxform (options=0x638c09689700, buffer=0x7169fc664120, det_ctx=<optimised out>) at detect-transform-luaxform.c:336
#11 TransformLuaxform (det_ctx=<optimised out>, buffer=0x7169fc664120, options=0x638c09689700) at detect-transform-luaxform.c:305
#12 0x0000638be8dc2927 in InspectionBufferApplyTransformsInternal (transforms=<optimised out>, buffer=<optimised out>, det_ctx=<optimised out>) at detect-engine-inspect-buffer.c:122
#13 InspectionBufferApplyTransformsInternal (transforms=0x638c0968d638, buffer=0x7169fc664120, det_ctx=0x7169fc3cae30) at detect-engine-inspect-buffer.c:113
#14 InspectionBufferSetupMulti (det_ctx=det_ctx@entry=0x7169fc3cae30, buffer=buffer@entry=0x7169fc664120, transforms=transforms@entry=0x638c0968d638, data=<optimised out>, data_len=<optimised out>) at detect-engine-inspect-buffer.c:168
#15 0x0000638be8d0994f in FiledataWithXformsGetDataCallback (base_buffer=0x7169fc665130, local_file_id=0, list_id=415, transforms=0x638c0968d638, det_ctx=0x7169fc3cae30) at detect-file-data.c:280
#16 FiledataGetDataCallback (det_ctx=det_ctx@entry=0x7169fc3cae30, transforms=0x638c0968d638, f=f@entry=0x638c09510ae0, flow_flags=flow_flags@entry=8 '\b', cur_file=cur_file@entry=0x7169fc65ad10, list_id=list_id@entry=415, base_id=191, local_file_id=0, 
    txv=0x7169fc669f30) at detect-file-data.c:467
#17 0x0000638be8d09df6 in PrefilterTxFiledata (p=<optimised out>, idx=<optimised out>, txd=0x7169fc668230, flags=<optimised out>, txv=0x7169fc669f30, f=0x638c09510ae0, pectx=0x638c0968af50, det_ctx=0x7169fc3cae30) at detect-file-data.c:555
#18 PrefilterTxFiledata (det_ctx=0x7169fc3cae30, pectx=0x638c0968af50, p=<optimised out>, f=0x638c09510ae0, txv=0x7169fc669f30, idx=<optimised out>, txd=0x7169fc668230, flags=8 '\b') at detect-file-data.c:538
#19 0x0000638be8cf8c38 in DetectRunPrefilterTx (det_ctx=det_ctx@entry=0x7169fc3cae30, sgh=sgh@entry=0x638c0968c460, p=p@entry=0x7169fc28c140, ipproto=ipproto@entry=6 '\006', flow_flags=flow_flags@entry=8 '\b', alproto=alproto@entry=2, 
    alstate=0x7169fc3da8a0, tx=0x716a012fbf00) at detect-engine-prefilter.c:149
#20 0x0000638be8d3dba9 in DetectRunTx (scratch=0x716a012fbed0, f=<optimised out>, p=<optimised out>, det_ctx=0x0, de_ctx=<optimised out>, tv=0x7169fc669f30) at detect.c:1761
#21 DetectRun (th_v=th_v@entry=0x638c0968c8d0, de_ctx=de_ctx@entry=0x638c095ae620, det_ctx=det_ctx@entry=0x7169fc3cae30, p=p@entry=0x7169fc28c140) at detect.c:190
#22 0x0000638be8d3e985 in DetectNoFlow (p=<optimised out>, det_ctx=<optimised out>, de_ctx=<optimised out>, tv=<optimised out>) at detect.c:2307
#23 Detect (tv=tv@entry=0x638c0968c8d0, p=p@entry=0x7169fc28c140, data=data@entry=0x7169fc3cae30) at detect.c:2387
#24 0x0000638be8d45ede in FlowWorker (tv=0x638c0968c8d0, p=0x7169fc28c140, data=0x7169fc2caef0) at flow-worker.c:669
#25 0x0000638be8ca2497 in TmThreadsSlotVarRun (tv=tv@entry=0x638c0968c8d0, p=p@entry=0x7169fc28c140, slot=<optimised out>) at tm-threads.c:137
#26 0x0000638be8d641a5 in TmThreadsSlotProcessPkt (p=0x7169fc28c140, s=<optimised out>, tv=0x638c0968c8d0) at /home/chris/src/git/suricata/src/tm-threads.h:201
#27 PcapFileCallbackLoop (user=0x7169fc28cbf0 "P\314*\374iq", h=0x716a012fc130, pkt=<optimised out>) at source-pcap-file-helper.c:108
#28 0x0000716a0330c63f in ?? () from /lib/x86_64-linux-gnu/libpcap.so.0.8
#29 0x0000638be8d644f9 in PcapFileDispatch (ptv=0x7169fc28cbf0) at source-pcap-file-helper.c:153
#30 0x0000638be8d654d3 in ReceivePcapFileLoop (tv=<optimised out>, data=0x7169fc28cb70, slot=<optimised out>) at source-pcap-file.c:207
#31 0x0000638be8ca3f5e in TmThreadsSlotPktAcqLoop (td=0x638c0968c8d0) at tm-threads.c:334
#32 0x0000716a0249caa4 in start_thread (arg=<optimised out>) at ./nptl/pthread_create.c:447
#33 0x0000716a02529c6c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

which suggests it's something to do with this "settable" instruction in `detect-transform-luaxform.c`:

    /* Lua script args are: buffer, rule args table */
    LuaPushStringBuffer(tlua->luastate, input, (size_t)input_len);
    /*
     * Add provided arguments for lua script (these are optionally
     * provided by the rule writer).
     *
     * Start at offset 1 (arg[0] is the lua script filename)
     */
    lua_newtable(tlua->luastate);
    for (int i = 1; i < lua->arg_count + 1; i++) {
        LuaPushInteger(tlua->luastate, i);
        lua_pushstring(tlua->luastate, lua->args[i]);
        lua_settable(tlua->luastate, -3);
    }

Tested 8.0.1 release and latest 9.0.0-dev. Other PCAPs tested work fine, so quite what's in this traffic to trigger the bug is a mystery.