Suricata

It seems if there is a syntax error in a flow/xbit rule so that it does not load and the rule uses lua on a sticky buffer, I can consistently reproduce the following:

The rule (think this can be reproduced with any lua script)

alert http any any -> any any (msg:"test"; file.name; content:".exe"; endswith; lua:any.lua; xbits: isset, mychemical.romance; track ip_dst, expire 3600; sid:10; rev:1;)

sudo /opt/suritest/bin/suricata -S test.rules -l log/ -k none -r /home/pevma/inthetrenches/tmp/test.pcap 
[21804] 22/10/2019 -- 22:39:07 - (suricata.c:1071) <Notice> (LogVersion) -- This is Suricata version 5.0.0-dev (64a789bbf 2019-10-18) running in USER mode
[21804] 22/10/2019 -- 22:39:08 - (detect-xbits.c:205) <Error> (DetectXbitParse) -- [ERRCODE: SC_ERR_PCRE_MATCH(2)] - "isset, mychemical.romance" is not a valid setting for xbits.
[21804] 22/10/2019 -- 22:39:08 - (detect-engine-loader.c:184) <Error> (DetectLoadSigFile) -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http any any -> any any (msg:"test"; file.name; content:".exe"; endswith; lua:emotet.lua; xbits: isset, mychemical.romance; track ip_dst, expire 3600; sid:10; rev:1;)" from file test.rules at line 1
[21804] 22/10/2019 -- 22:39:08 - (detect-engine-loader.c:345) <Warning> (SigLoadSignatures) -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 1 rule files specified, but no rule was loaded at all!
=================================================================
==11773==ERROR: AddressSanitizer: heap-use-after-free on address 0x60f000028af4 at pc 0x55c2a91fd37f bp 0x7f470703c670 sp 0x7f470703c668
READ of size 2 at 0x60f000028af4 thread T2 (W#01)
    #0 0x55c2a91fd37e in DetectLuaThreadInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-lua.c:596
    #1 0x55c2a90e6735 in DetectEngineThreadCtxInitKeywords /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine.c:2509
    #2 0x55c2a90eb0bd in ThreadCtxDoInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine.c:2732
    #3 0x55c2a90eb943 in DetectEngineThreadCtxInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine.c:2788
    #4 0x55c2a92b9122 in FlowWorkerThreadInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/flow-worker.c:127
    #5 0x55c2a94cb33f in TmThreadsSlotVar /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/tm-threads.c:541
    #6 0x7f471c041fb6 in start_thread /build/glibc-sPWrSm/glibc-2.29/nptl/pthread_create.c:486
    #7 0x7f471b3c42ee in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfa2ee)

0x60f000028af4 is located 20 bytes inside of 168-byte region [0x60f000028ae0,0x60f000028b88)
freed by thread T0 (Suricata-Main) here:
    #0 0x7f471ca00187 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x107187)
    #1 0x55c2a9204e1b in DetectLuaFree /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-lua.c:1122
    #2 0x55c2a9214f88 in SigMatchFree /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:255
    #3 0x55c2a9220cff in SigFree /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:1396
    #4 0x55c2a9228dec in SigInitHelper /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:1962
    #5 0x55c2a9229319 in SigInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:2018
    #6 0x55c2a922b32f in DetectEngineAppendSig /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:2304
    #7 0x55c2a9140340 in DetectLoadSigFile /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:169
    #8 0x55c2a9141616 in ProcessSigFiles /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:250
    #9 0x55c2a9142484 in SigLoadSignatures /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:329
    #10 0x55c2a94b862c in LoadSignatures /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:2472
    #11 0x55c2a94b9eb7 in PostConfLoadedDetectSetup /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:2626
    #12 0x55c2a94bd9ca in main /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:3051
    #13 0x7f471b2f0bba in __libc_start_main ../csu/libc-start.c:308

previously allocated by thread T0 (Suricata-Main) here:
    #0 0x7f471ca00538 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x107538)
    #1 0x55c2a91fe0f1 in DetectLuaParse /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-lua.c:673
    #2 0x55c2a920384e in DetectLuaSetup /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-lua.c:978
    #3 0x55c2a921bc82 in SigParseOptions /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:802
    #4 0x55c2a921ec69 in SigParse /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:1242
    #5 0x55c2a9228382 in SigInitHelper /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:1866
    #6 0x55c2a9229319 in SigInit /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:2018
    #7 0x55c2a922b32f in DetectEngineAppendSig /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-parse.c:2304
    #8 0x55c2a9140340 in DetectLoadSigFile /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:169
    #9 0x55c2a9141616 in ProcessSigFiles /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:250
    #10 0x55c2a9142484 in SigLoadSignatures /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-engine-loader.c:329
    #11 0x55c2a94b862c in LoadSignatures /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:2472
    #12 0x55c2a94b9eb7 in PostConfLoadedDetectSetup /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:2626
    #13 0x55c2a94bd9ca in main /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:3051
    #14 0x7f471b2f0bba in __libc_start_main ../csu/libc-start.c:308

Thread T2 (W#01) created by T0 (Suricata-Main) here:
    #0 0x7f471c9329b2 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x399b2)
    #1 0x55c2a94d426f in TmThreadSpawn /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/tm-threads.c:1868
    #2 0x55c2a93eb60a in RunModeFilePcapAutoFp /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/runmode-pcap-file.c:252
    #3 0x55c2a93fc387 in RunModeDispatch /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/runmodes.c:377
    #4 0x55c2a94bdc05 in main /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/suricata.c:3060
    #5 0x7f471b2f0bba in __libc_start_main ../csu/libc-start.c:308

SUMMARY: AddressSanitizer: heap-use-after-free /home/pevma/inthetrenches/Suricata/suricomp/suricata/src/detect-lua.c:596 in DetectLuaThreadInit
Shadow bytes around the buggy address:
  0x0c1e7fffd100: fa fa 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c1e7fffd110: 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa fa
  0x0c1e7fffd120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c1e7fffd130: 00 00 00 00 00 fa fa fa fa fa fa fa fa fa fd fd
  0x0c1e7fffd140: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c1e7fffd150: fd fd fd fa fa fa fa fa fa fa fa fa fd fd[fd]fd
  0x0c1e7fffd160: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c1e7fffd170: fd fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd
  0x0c1e7fffd180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c1e7fffd190: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c1e7fffd1a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==11773==ABORTING

This is Suricata version 5.0.0-dev (64a789bbf 2019-10-18)
Features: PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS MAGIC RUST 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrinsics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 9.2.1 20190909, C version 199901
compiled with -fstack-protector-all
compiled with _FORTIFY_SOURCE=0
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.31, linked against LibHTP v0.5.31

Suricata Configuration:
  AF_PACKET support:                       yes
  eBPF support:                            no
  XDP support:                             no
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no 
  DAG enabled:                             no
  Napatech enabled:                        no
  WinDivert enabled:                       no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  Libmagic support:                        yes
  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  hiredis support:                         no
  hiredis async with libevent:             no
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  GeoIP2 support:                          yes
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  Hyperscan support:                       yes
  Libnet support:                          yes
  liblz4 support:                          yes

  Rust support:                            yes
  Rust strict mode:                        yes
  Rust compiler path:                      /usr/bin/rustc
  Rust compiler version:                   rustc 1.37.0
  Cargo path:                              /usr/bin/cargo
  Cargo version:                           cargo 1.36.0

  Python support:                          yes
  Python path:                             /usr/bin/python3
  Python distutils                         yes
  Python yaml                              yes
  Install suricatactl:                     yes
  Install suricatasc:                      yes
  Install suricata-update:                 yes

  Profiling enabled:                       no
  Profiling locks enabled:                 no

Development settings:
  Coccinelle / spatch:                     yes
  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no

Generic build parameters:
  Installation prefix:                     /opt/suritest
  Configuration directory:                 /opt/suritest/etc/suricata/
  Log directory:                           /opt/suritest/var/log/suricata/

  --prefix                                 /opt/suritest
  --sysconfdir                             /opt/suritest/etc
  --localstatedir                          /opt/suritest/var
  --datarootdir                            /opt/suritest/share

  Host:                                    x86_64-pc-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -ggdb3 -Werror -Wchar-subscripts -fno-strict-aliasing -fstack-protector-all -fsanitize=address -fno-omit-frame-pointer -Wno-unused-parameter -Wno-unused-function -march=native -I${srcdir}/../rust/gen/c-headers
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS