Project

General

Profile

Actions

Bug #2190

closed

apparent 1000 character limit in threshold.conf IP lists

Added by Duane Howard about 4 years ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:
Needs backport to 6.0

Description

something like:
suppress gen_id 0, sig_id 0, track by_src, ip [lots of chars] (see [0])
will elicit an error:
<Error> - [ERRCODE: SC_ERR_PCRE_GET_SUBSTRING(4)] - pcre_copy_substring failed

shortening this list (see [1]) to less than 1000 characters, will allow it to be successfully loaded.
Workarounds include moving list to a variable in suricata.yaml or splitting into chunks.

[0] Generate a broken list (too long)
$ echo -ne '['; for in in $(seq 1 53); do echo -n '123.123.123.123/32,'; done | head -c -1; echo ']'
[123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32]

[1] Generate a working list (small enough)
$ echo -ne '['; for in in $(seq 1 52); do echo -n '123.123.123.123/32,'; done | head -c -1; echo ']'
[123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32,123.123.123.123/32]


Related issues

Related to Feature #3887: yaml: Increase maximum size for address varsClosedJeff LucovskyActions
Related to Task #4097: Suricon 2020 brainstormNewVictor JulienActions
Has duplicate Bug #4377: 64 Character Limit on IP for thresholding.confClosedActions
Copied to Bug #4416: apparent 1000 character limit in threshold.conf IP listsClosedShivani BhardwajActions
Actions #1

Updated by Andreas Herz about 4 years ago

  • Assignee set to OISF Dev
  • Target version set to TBD

So we either improve the error message or increase the limit. Will be interesting with more IPv6 usage, so I think increasing it and document the limit might be the best.

Actions #2

Updated by Andreas Herz about 2 years ago

Any thoughts on my suggestion?

Actions #3

Updated by Victor Julien about 2 years ago

The only internal limit I see is 8k for a max line length. Is the 1000 chars limit still current?

Actions #4

Updated by Andreas Herz about 2 years ago

yep I still get this:

[15850] 23/7/2019 -- 22:37:27 - (util-threshold-config.c:734) <Error> (ParseThresholdRule) -- [ERRCODE: SC_ERR_PCRE_GET_SUBSTRING(4)] - pcre_copy_substring failed

But also
[15850] 23/7/2019 -- 22:37:27 - (util-threshold-config.c:1126) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 1 rule(s) found

Actions #5

Updated by Jason Ish 10 months ago

  • Related to Feature #3887: yaml: Increase maximum size for address vars added
Actions #6

Updated by Jason Ish 10 months ago

  • Related to Task #4097: Suricon 2020 brainstorm added
Actions #7

Updated by Victor Julien 10 months ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Jeff Lucovsky
  • Target version changed from TBD to 7.0rc1
Actions #8

Updated by Jeff Lucovsky 10 months ago

  • Status changed from Assigned to In Review
Actions #9

Updated by Jeff Lucovsky 6 months ago

  • Has duplicate Bug #4379: flow manager: using too much CPU during idle added
Actions #10

Updated by Jeff Lucovsky 6 months ago

  • Has duplicate Bug #4377: 64 Character Limit on IP for thresholding.conf added
Actions #11

Updated by Jeff Lucovsky 6 months ago

  • Label Needs backport to 6.0 added
Actions #12

Updated by Jeff Lucovsky 5 months ago

  • Copied to Bug #4416: apparent 1000 character limit in threshold.conf IP lists added
Actions #13

Updated by Victor Julien 5 months ago

  • Has duplicate deleted (Bug #4379: flow manager: using too much CPU during idle)
Actions #14

Updated by Jeff Lucovsky 5 months ago

  • Status changed from In Review to Closed
Actions

Also available in: Atom PDF