Project

General

Profile

Actions

Task #3194

closed

pcre2 support

Added by Victor Julien over 4 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Upstream pcre developers are working on pcre2 and have been for a while. I would like to know the impact on Suricata of supporting this.

- can they coexist in a single program (we use pcre for match and parsing)
- what does distro support look like for 'tier 1' in Support Status
- are there features we use that are missing from pcre2?
- estimate of effort for just supporting it for matching
- estimate of effort for completely replacing it


Subtasks 1 (0 open1 closed)

Task #4446: pcre2: document changes vs prce1 for rule writersClosedPhilippe AntoineActions
Actions #1

Updated by Victor Julien over 3 years ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Jeff Lucovsky
  • Target version changed from 70 to 7.0.0-beta1
Actions #2

Updated by Philippe Antoine over 3 years ago

Is the goal of this ticket to replace pcre by pcre2 ?
If so, it will be useful for fuzzing, as it finds bugs in pcre which will likely not be fixed because pcre is at end of life and all the development is now in pcre2
cf https://bugs.exim.org/show_bug.cgi?id=2661

Actions #3

Updated by Victor Julien over 3 years ago

Ultimately yes. I would like us to get away from using pcre for rule/config parsing, so that we only use it for the "pcre" rule keyword.

Actions #4

Updated by Philippe Antoine over 3 years ago

Ok, so this is about rustifying all input handling, right ?

From the fuzzing point of view, it is useful to replace pcre by pcre2 even as long as we keep it for rule/config parsing

Actions #6

Updated by Philippe Antoine about 3 years ago

- can they coexist in a single program (we use pcre for match and parsing)

I do not see why not
I do not see the point though, but I neither see the difference between supporting it for matching and completely replacing it
Is it just using pcre2 in detect-pcre*.c ?

what does distro support look like for 'tier 1' in Support Status

I guess it should be pretty good.
PCRE1 (the 8.xx series) is at end of life.
Should CI check this ?

estimate of effort for just supporting it for matching
estimate of effort for completely replacing it

I do not see a difference between these two

Used methods seem
- pcre_free_substring replaced straight by pcre2_substring_free
- pcre_get_substring seems replaced by pcre2_substring_get_bynumber which does not have the same arguments...
- pcre_exec replaced by pcre2_match
- pcre_copy_substring replaced by pcre2_substring_copy_bynumber
- pcre_free
- pcre_compile replaced by pcre2_compile
- pcre_study obsolete in pcre2 : The new API is more extensible, and it was simplified by abolishing the separate "study" optimizing function
- pare jit stuff present in pcre2

To investigate : the suricata.yaml pcre.match-limit and match-limit-recursion
Seems like it is supported with PCRE2_CONFIG_DEPTHLIMIT

Looks like a few days work to get something testable
And then the work needed to test it...

are there features we use that are missing from pcre2?

It looks complete, needs to be tested...
pcre_study is now obsolete but the features must be there

Actions #7

Updated by Philippe Antoine about 3 years ago

For reference : https://github.com/catenacyber/suricata/tree/pcre2-3194-v1

Incomplete work : needs pcre_get_substring and pcre_copy_substring to be translated...

Actions #8

Updated by Victor Julien about 3 years ago

CentOS7 has pcre2 support in base, version is 10.23-2.el7. We should make sure that works with our code.

Actions #9

Updated by Victor Julien about 3 years ago

  • Subject changed from investigate pcre2 support to pcre2 support
  • Status changed from Assigned to In Progress
  • Assignee changed from Jeff Lucovsky to Philippe Antoine
Actions #11

Updated by Philippe Antoine almost 3 years ago

  • Status changed from In Progress to In Review
Actions #12

Updated by Philippe Antoine over 2 years ago

  • Status changed from In Review to Closed
Actions #13

Updated by Sascha Steinbiss over 2 years ago

FYI https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=999952 <- Moving to PCRE2 over the deprecated old PCRE has also been suggested by Debian

Actions #14

Updated by Philippe Antoine over 2 years ago

Thanks Sasha for answering there

Actions #15

Updated by Victor Julien over 2 years ago

@Sascha Steinbiss Is a backport of this work desired? Its somewhat intrusive, but not impossible of course.

Actions #16

Updated by Sascha Steinbiss over 2 years ago

Victor Julien wrote in #note-15:

@Sascha Steinbiss Is a backport of this work desired? Its somewhat intrusive, but not impossible of course.

Well, if you think it's going to be a while until Suri 7 is released, then a backport would be appreciated in the meantime. If you think it will be released in the coming few months, then I don't think so.
Obviously, the sooner we can switch the dependency, the better. But AFAICS the goal is to retire the old PCRE library in Debian before the Bookworm freeze, which is not even planned yet.

Actions

Also available in: Atom PDF