Project

General

Profile

Actions

Bug #1577

closed

luajit scripts load error

Added by Samiux A over 8 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

I am using the latest git version - Suricata version 2.1dev (rev dcbbda5). I am using these rules (https://github.com/EmergingThreats/et-luajit-scripts/blob/master/luajit-drop.rules) from https://github.com/EmergingThreats/et-luajit-scripts

The build info is :

suricata --build-info
This is Suricata version 2.1dev (rev dcbbda5)
Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS 
SIMD support: SSE_4_2 SSE_4_1 SSE_3 
Atomic intrisics: 1 2 4 8 16 byte(s)
64-bits, Little-endian architecture
GCC version 4.8.4, C version 199901
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.18, linked against LibHTP v0.5.18

Suricata Configuration:
  AF_PACKET support:                       yes
  PF_RING support:                         no
  NFQueue support:                         no
  NFLOG support:                           no
  IPFW support:                            no
  Netmap support:                          no
  DAG enabled:                             no
  Napatech enabled:                        no

  Unix socket enabled:                     yes
  Detection enabled:                       yes

  libnss support:                          yes
  libnspr support:                         yes
  libjansson support:                      yes
  Prelude support:                         no
  PCRE jit:                                yes
  LUA support:                             yes, through luajit
  libluajit:                               yes
  libgeoip:                                yes
  Non-bundled htp:                         no
  Old barnyard2 support:                   no
  CUDA enabled:                            no

  Suricatasc install:                      yes

  Unit tests enabled:                      no
  Debug output enabled:                    no
  Debug validation enabled:                no
  Profiling enabled:                       no
  Profiling locks enabled:                 no
  Coccinelle / spatch:                     no

Generic build parameters:
  Installation prefix:                     /usr
  Configuration directory:                 /etc/suricata/
  Log directory:                           /var/log/suricata/

  --prefix                                 /usr
  --sysconfdir                             /etc
  --localstatedir                          /var

  Host:                                    x86_64-unknown-linux-gnu
  Compiler:                                gcc (exec name) / gcc (real)
  GCC Protect enabled:                     no
  GCC march native enabled:                yes
  GCC Profile enabled:                     no
  Position Independent Executable enabled: no
  CFLAGS                                   -g -O2 -march=native
  PCAP_CFLAGS                               -I/usr/include
  SECCFLAGS                            

The suricata.yaml setting is :
  # Lua Output Support - execute lua script to generate alert and event
  # output.
  # Documented at:
  # https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Lua_Output
  - lua:
#      enabled: no
      enabled: yes
      #scripts-dir: /etc/suricata/lua-output/
      scripts-dir: /etc/suricata/rules/
      scripts:
      #   - script1.lua
#          - CVE-2012-1535.lua
#          - CVE-2013-0074.lua
#          - CVE-2014-4114.lua
#          - CVE-2015-1641.lua
#          - CVE-2015-1650.lua
#          - CVE-2015-1770.lua
#          - CVE-2015-2375.lua
#          - CVE-2015-2377.lua
#          - CVE-2015-2426.lua
#          - CVE-2015-3113.lua
#          - suri-bh2-abc-jar.lua
#          - suri-high-entropy.lua
#          - suri-nuclear-url.lua
#          - suri-regin.lua
#          - suri-reversed-compressed-binary.lua
#          - suri-styx-url.lua
#          - suri-suspicious-flash2.lua
#          - suri-suspicious-jar.lua
#          - suri-suspicious-jar2.lua
#          - suri-suspicious-pack200jar.lua
#          - suri-suspicious-pdf.lua
#          - suri-suspicious-vbe.lua
#          - suri-xor-binary-detect.lua
#          - suri-xor-binary-quick.lua
#          - suri-xor-non-zero.lua
#          - tls-heartbleed.lua

All the captioned lua script dependencies have been solved. All the said lua scripts are placed at /etc/suricata/rules/. When I restart Suricata, I have the following error at /var/log/suricata.log :

http://pastebin.com/7PQ4KbsP

Meanwhile, when I uncomment the suricata.yaml for the captioned lua scripts at the lua section, the Suricata restart itself in an endless loop condition.

Actions #1

Updated by Samiux A over 8 years ago

For best view, please read http://pastebin.com/HKi8e7XU

Actions #2

Updated by Mats Klepsland over 8 years ago

Those are lua detection scripts, not lua output scripts, so they should not be added to the suricata.yaml. They should only be used in rules using the "luajit" keyword.

Actions #3

Updated by Victor Julien over 8 years ago

  • Description updated (diff)
  • Priority changed from High to Normal
Actions #4

Updated by Victor Julien over 8 years ago

Seems detection and output scripts are getting mixed up here. The lua support under the output section is for output/logging scripts. The detection scripts shouldn't be loaded here.

Actions #5

Updated by Samiux A over 8 years ago

Where can I place the lua detection scripts at?

I tried to put them at /etc/suricata/rules and the lua section at suricata.yaml has been disabled, the lua related rules cannot be loaded properly.

What I missed?

Actions #6

Updated by Samiux A over 8 years ago

The problem is solved when I change the value of "prealloc" at "flow" section back to default (10000).

flow:
memcap: 16gb
hash-size: 65536
prealloc: 10000
  1. prealloc: 10485760

How large the value of "prealloc" can be set?

Actions #7

Updated by Victor Julien over 8 years ago

  • Status changed from New to Assigned
  • Assignee set to Victor Julien
  • Target version set to 3.0RC1

Looks like another case where luajit pools get setup too late. It needs its memory in the below 2GiB range iirc. I'll see if I can make it initialize a bit earlier.

Actions #8

Updated by Victor Julien over 8 years ago

  • Target version changed from 3.0RC1 to 70
Actions #9

Updated by kevin ross almost 8 years ago

Hi,

I also have an issue with this where luajit sigs do not load in. I have changed the prealloc stuff to recommended values + really high values and it does not work. Is there any other information needed?

20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] - luastate pool depleted
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] - setting up thread local detect ctx for keyword "luajit" failed
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] - luastate pool depleted
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] - setting up thread local detect ctx for keyword "luajit" failed
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] - luastate pool depleted
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] - setting up thread local detect ctx for keyword "luajit" failed
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] - luastate pool depleted
20/4/2016 -- 12:09:47 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] - setting up thread local detect ctx for keyword "luajit" failed

Kind Regards,
Kevin

Here is my new build options:
  1. suricata --build-info
    This is Suricata version 3.0.1 RELEASE
    Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON PROFILING TLS
    SIMD support: SSE_4_2 SSE_4_1 SSE_3
    Atomic intrisics: 1 2 4 8 16 byte(s)
    64-bits, Little-endian architecture
    GCC version 4.8.5 20150623 (Red Hat 4.8.5-4), C version 199901
    compiled with _FORTIFY_SOURCE=0
    L1 cache line size (CLS)=64
    thread local storage method: __thread
    compiled with LibHTP v0.5.19, linked against LibHTP v0.5.19

Suricata Configuration:
AF_PACKET support: yes
PF_RING support: yes
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no

Unix socket enabled:                     yes
Detection enabled: yes
libnss support:                          yes
libnspr support: yes
libjansson support: yes
hiredis support: no
Prelude support: no
PCRE jit: yes
LUA support: yes, through luajit
libluajit: yes
libgeoip: yes
Non-bundled htp: no
Old barnyard2 support: no
CUDA enabled: no
Hyperscan support: yes
Actions #10

Updated by kevin ross almost 8 years ago

by any more information I mean from me for testing/troubleshooting

Actions #11

Updated by Victor Julien over 7 years ago

  • Status changed from Assigned to Closed
  • Target version changed from 70 to 3.2
Actions #12

Updated by Victor Julien over 7 years ago

  • Subject changed from lua scripts load error in 2.1dev (rev dcbbda5) to luajit scripts load error
Actions

Also available in: Atom PDF