Project

General

Profile

Actions

Feature #6831

open
BM VJ

rules: support extraction of bytes of non-numeric values

Feature #6831: rules: support extraction of bytes of non-numeric values

Added by Brandon Murphy about 2 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

Use Case:

Consider the following HTTP request. I would like to write detection logic that ensures the 32 byte string that occurs in the URI is also found within the Cookie Value.

GET /example/path?foo=Pn5tRZrj12eSWFx4qL7cAeyzvKl90O6G HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:97.0) Gecko/20100101 Firefox/97.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://example.com
Cookie: foo=Pn5tRZrj12eSWFx4qL7cAeyzvKl90O6G

reference: https://forum.suricata.io/t/byte-extract-byte-test-string-limits/4511/3

Current and Attempted Methods:

Currently the only way to accomplish this detection logic (with PCRE capture groups) has many limitations/considerations
1) within a single buffer (http.start works in this example)
- this won't work with HTTP/2 traffic
2) via capture groups within PCRE. This can be pretty costly.

I attempted to use the byte_extract/byte_test, which did work, but was limited to 20 byte as noted within the discourse convo. the keyword performance of the byte_extract/byte_test combo was about half as many ticks as using the PCRE capture groups.

Proposed Solution:
I won't bother with suggesting how this should be solved, but more details of the use case are required please let me know.

P.S. - I'm not sure what to "call" this request, so feel free to change the subject to more accurately reflect whatever this is.


Related issues 4 (4 open0 closed)

Related to Suricata - Feature #7321: rules: cross buffer byte_* keyword supportNewOISF DevActions
Related to Suricata - Task #7336: Suricon 2024 brainstormAssignedVictor JulienActions
Related to Suricata - Task #8123: Suricon 2025 BrainstormAssignedVictor JulienActions
Related to Suricata - Feature #7847: rules: extend byte_extract named variables for use in other keywords/transformations such as xorIn ReviewJeff LucovskyActions

BM Updated by Brandon Murphy about 2 years ago Actions #1

  • Description updated (diff)

BM Updated by Brandon Murphy about 2 years ago Actions #2

  • Description updated (diff)

JL Updated by Jeff Lucovsky about 2 years ago Actions #3

@zoomequipd Can you add the rule demonstrating the use of pcre?

Did you try the pcrexform transform?

BM Updated by Brandon Murphy about 2 years ago Actions #4

Jeff Lucovsky wrote in #note-3:

@zoomequipd Can you add the rule demonstrating the use of pcre?

you bet! Something like this currently "does the trick", though as noted

http.start; pcre:"/^POST[^\r\n]+\?foo=(?P<hash>[A-F0-9]{32})(?:[^\r\n]*\r\n)+Cookie\x3a\x20foo=(?P=hash)/";

Did you try the pcrexform transform?

I'm not sure it'd solve the issue given the need to enforce that the content exist in two different buffers. I guess i could use it to transform each buffer to the be same, but i'd still be left with the need to extract the value and compare.

JL Updated by Jeff Lucovsky about 2 years ago Actions #5

For completeness, this is what was presented in the discourse forum:

http.uri;  content:"&foo="; byte_extract:32,0,TESTrelative,string; http.cookie; content:"foo="; startswith; byte_test:32,=,TEST,0,relative,string;

VJ Updated by Victor Julien about 2 years ago Actions #6

Jeff Lucovsky wrote in #note-5:

For completeness, this is what was presented in the discourse forum:
[...]

I like the idea of using regular byte_extract and byte_test. The "string" option in the example makes no sense to me. It should probably be called something else, like "bytes" or "slice" or "raw" or similar, to indicate that we won't try to interpret the bytes as an integer but instead will do a raw byte slice compare (a memcmp essentially).

VJ Updated by Victor Julien over 1 year ago Actions #7

  • Related to Feature #7321: rules: cross buffer byte_* keyword support added

VJ Updated by Victor Julien over 1 year ago Actions #8

  • Related to Task #7336: Suricon 2024 brainstorm added

VJ Updated by Victor Julien 5 months ago Actions #9

  • Subject changed from support extraction of bytes of non-numeric values to rules: support extraction of bytes of non-numeric values

JF Updated by Juliana Fajardini Reichow 5 months ago Actions #10

  • Related to Task #8123: Suricon 2025 Brainstorm added

JF Updated by Juliana Fajardini Reichow 5 months ago Actions #11

As of SuriCon 2025's Brainstorm, still of interest to rule writers.

VJ Updated by Victor Julien 5 months ago Actions #12

  • Assignee changed from OISF Dev to Victor Julien
  • Target version changed from TBD to 9.0.0-beta1

I would like to see this done in a generic way for all scopes we have vars/bits/etc for, and probably more:
- rule
- packet
- transaction
- flow
- host
- ippair

Where it makes sense it should hook into the existing things we do, like extracting bytes with pcre substring capture.

VJ Updated by Victor Julien 4 months ago Actions #13

  • Related to Feature #7847: rules: extend byte_extract named variables for use in other keywords/transformations such as xor added
Actions

Also available in: PDF Atom