Project

General

Profile

Actions

Feature #6487

open

transform: from_base64

Added by Jason Ish 6 months ago. Updated about 22 hours ago.

Status:
In Review
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:


Related issues 4 (4 open0 closed)

Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Bug #5220: fast_pattern specification in base64_data shouldn't be allowedIn ReviewShivani BhardwajActions
Related to Suricata - Feature #5245: allow fast_pattern on base64_data stringsNewOISF DevActions
Precedes Suricata - Feature #6807: Support the use of variables within transformsNewOISF DevActions
Actions #1

Updated by Jason Ish 6 months ago

  • Related to Task #6443: Suricon 2023 brainstorm added
Actions #2

Updated by Brandon Murphy 6 months ago

just want to add that, as victor touch on in this comment https://github.com/OISF/suricata/pull/7233#issuecomment-1099277048

the transform should support some method of "selecting" which bytes to decode.

Shivani has done a lot of work recently getting the base64_decode to handle errors in a way that is reasonable. We'd want to make sure that same behavior occurs here.
References:
#5896 and #5223 and #6135

Also, while I"m sure this is a long stretch, but if i recall correctly transformations can still be used as fast_patterns (from_base64 might be an exception?) but if this transforms opens a path to seeing #5245 implemented that would be amazing!

Actions #3

Updated by Victor Julien 5 months ago

  • Status changed from New to In Progress
  • Assignee changed from OISF Dev to Jeff Lucovsky
  • Target version changed from TBD to 8.0.0-beta1

Needs a way to specify the data to decode. An offset and length at least. Defaulting to all. Wonder if it makes sense to allow specifying the parsing mode. From util-base64.h we'd have: relax, rfc2045, strict, rfc4648.

Actions #4

Updated by Jeff Lucovsky 5 months ago

from_base64: [bytes <n>, offset <n>, mode <relax|rf2045|strict|rfc4648>]

The bytes and offset default to inspection buffer length and 0, respectively.

mode can default to that used with base64_decode -- RFC4648.

Thoughts?

Actions #5

Updated by Victor Julien 5 months ago

Should we also support variables here? From byte_extract I mean?

Actions #6

Updated by Jeff Lucovsky 5 months ago

That's a good idea. If it's possible to use a variable for the offset and bytes value, we should.

Actions #7

Updated by Victor Julien 4 months ago

  • Related to Bug #5220: fast_pattern specification in base64_data shouldn't be allowed added
Actions #8

Updated by Victor Julien 4 months ago

  • Related to Feature #5245: allow fast_pattern on base64_data strings added
Actions #9

Updated by Jeff Lucovsky about 2 months ago

  • Status changed from In Progress to In Review
Actions #10

Updated by Jeff Lucovsky about 2 months ago

  • Precedes Feature #6807: Support the use of variables within transforms added
Actions #11

Updated by Jeff Lucovsky about 2 months ago

We've deferred using byte variables (see issue 6807).

Actions #12

Updated by Jeff Lucovsky about 2 months ago

@Brandon Murphy

Can you comment on which base64 decode modes are useful for rule writers. There's RFC2045, RFC4648 and then Suricata's own strict

strict stops decoding and returns an error iff an invalid character is found in the bytes being decoded.

Actions #13

Updated by Brandon Murphy about 2 months ago

Jeff Lucovsky wrote in #note-12:

@Brandon Murphy

Can you comment on which base64 decode modes are useful for rule writers. There's RFC2045, RFC4648 and then Suricata's own strict

strict stops decoding and returns an error iff an invalid character is found in the bytes being decoded.

Hard to say, they all have their purpose. Are you asking to help select a default mode if one isn't provided?

I know @Shivani Bhardwaj did a lot of work to allow "extracting" more data from strings in base64 format that have invalid characters in them which has been useful. references: #5885 #5315 #5223

Actions #14

Updated by Jeff Lucovsky about 22 hours ago

[from a discord thread in #rule-lang-dev]
@Jeff Lucovsky asked: "We'd like input on what to do when the input buffer cannot be base64 decoded (e.g., the buffer isn't a valid base64 string)"

@zoomequipd's reply:
three* thoughts:
1) if the string cant be base64 decoded, IMO the rule should not alert.
taking from your example, consider this rule
alert http any any -> any any (msg:"from_base64: offset #1 [mode rfc4648]"; \
http.uri; content:"/?arg="; from_base64: offset 6; \
content:"thisisatest"; sid:2; rev:1;)

when the traffic encountered incldues the URI /?args=notabase64encodedstring

In this case, i'm not really sure how to best handle it internlly to suricata, but i wouldn't want an alert to trigger.

2) would the condition when the string can be partially base64 decoded (encounters a non-base64 string 40 chars into the base64 string) be considred when the input buffer cannot be base64 decoded?

reference: https://redmine.openinfosecfoundation.org/issues/5885

3) does it make sense to include a "relative" option on the transformation?

Actions

Also available in: Atom PDF