Project

General

Profile

Actions

Task #3194

open

pcre2 support

Added by Victor Julien almost 2 years ago. Updated 5 months ago.

Status:
In Review
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Upstream pcre developers are working on pcre2 and have been for a while. I would like to know the impact on Suricata of supporting this.

- can they coexist in a single program (we use pcre for match and parsing)
- what does distro support look like for 'tier 1' in Support Status
- are there features we use that are missing from pcre2?
- estimate of effort for just supporting it for matching
- estimate of effort for completely replacing it


Subtasks 1 (1 open0 closed)

Task #4446: pcre2: document changes vs prce1 for rule writersIn ReviewPhilippe AntoineActions
Actions #1

Updated by Victor Julien about 1 year ago

  • Status changed from New to Assigned
  • Assignee changed from OISF Dev to Jeff Lucovsky
  • Target version changed from 70 to 7.0rc1
Actions #2

Updated by Philippe Antoine 11 months ago

Is the goal of this ticket to replace pcre by pcre2 ?
If so, it will be useful for fuzzing, as it finds bugs in pcre which will likely not be fixed because pcre is at end of life and all the development is now in pcre2
cf https://bugs.exim.org/show_bug.cgi?id=2661

Actions #3

Updated by Victor Julien 11 months ago

Ultimately yes. I would like us to get away from using pcre for rule/config parsing, so that we only use it for the "pcre" rule keyword.

Actions #4

Updated by Philippe Antoine 11 months ago

Ok, so this is about rustifying all input handling, right ?

From the fuzzing point of view, it is useful to replace pcre by pcre2 even as long as we keep it for rule/config parsing

Actions #6

Updated by Philippe Antoine 7 months ago

- can they coexist in a single program (we use pcre for match and parsing)

I do not see why not
I do not see the point though, but I neither see the difference between supporting it for matching and completely replacing it
Is it just using pcre2 in detect-pcre*.c ?

what does distro support look like for 'tier 1' in Support Status

I guess it should be pretty good.
PCRE1 (the 8.xx series) is at end of life.
Should CI check this ?

estimate of effort for just supporting it for matching
estimate of effort for completely replacing it

I do not see a difference between these two

Used methods seem
- pcre_free_substring replaced straight by pcre2_substring_free
- pcre_get_substring seems replaced by pcre2_substring_get_bynumber which does not have the same arguments...
- pcre_exec replaced by pcre2_match
- pcre_copy_substring replaced by pcre2_substring_copy_bynumber
- pcre_free
- pcre_compile replaced by pcre2_compile
- pcre_study obsolete in pcre2 : The new API is more extensible, and it was simplified by abolishing the separate "study" optimizing function
- pare jit stuff present in pcre2

To investigate : the suricata.yaml pcre.match-limit and match-limit-recursion
Seems like it is supported with PCRE2_CONFIG_DEPTHLIMIT

Looks like a few days work to get something testable
And then the work needed to test it...

are there features we use that are missing from pcre2?

It looks complete, needs to be tested...
pcre_study is now obsolete but the features must be there

Actions #7

Updated by Philippe Antoine 7 months ago

For reference : https://github.com/catenacyber/suricata/tree/pcre2-3194-v1

Incomplete work : needs pcre_get_substring and pcre_copy_substring to be translated...

Actions #8

Updated by Victor Julien 7 months ago

CentOS7 has pcre2 support in base, version is 10.23-2.el7. We should make sure that works with our code.

Actions #9

Updated by Victor Julien 7 months ago

  • Subject changed from investigate pcre2 support to pcre2 support
  • Status changed from Assigned to In Progress
  • Assignee changed from Jeff Lucovsky to Philippe Antoine
Actions #11

Updated by Philippe Antoine 5 months ago

  • Status changed from In Progress to In Review
Actions

Also available in: Atom PDF