Project

General

Profile

Actions

Bug #2881

open

http.protocol parsing inaccuracy : accept spaces in URI

Added by chris lujan over 4 years ago. Updated 10 days ago.

Status:
In Review
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
low
Label:

Description

Request:

GET /uid=0(root) gid=0(root) groups=0(root)asdf HTTP/1.1
User-Agent: curl/7.29.0
Accept: */*

eve.json output:
"http":{"protocol":"gid=0(root) groups=0(root)asdf HTTP\/1.1"}

It appears that the http.protocol is matching too greedily with the space character and could use something like /\S+$/m instead.


Files

b8ee56effed96ba.pcap (467 Bytes) b8ee56effed96ba.pcap Brandon Murphy, 06/15/2023 04:53 PM

Related issues 3 (0 open3 closed)

Related to Suricata - Task #3479: libhtp 0.5.33 (4.1.x)ClosedPhilippe AntoineActions
Related to Suricata - Task #3922: libhtp 0.5.35ClosedPhilippe AntoineActions
Related to Suricata - Task #4667: libhtp 0.5.39ClosedVictor JulienActions
Actions #1

Updated by chris lujan over 4 years ago

Conversely, the http.url field is only matching up until the first space resulting in something like:

"http":{"url":"/uid=0(root)"}

which leads me to believe those fields are created by splitting the line by spaces.

Actions #2

Updated by Victor Julien over 4 years ago

  • Status changed from New to Assigned
  • Assignee set to Philippe Antoine
  • Target version set to TBD
Actions #3

Updated by Victor Julien over 4 years ago

I think uri's are not supposed to have spaces, but I think it would be good to address this anyway.

Actions #4

Updated by Philippe Antoine over 4 years ago

Thanks Chris.
Indeed, Uris are not supposed to have spaces, but the protocol field is even less supposed to have spaces.
So I think we can take the last space in the request line as the uri end, instead of the second one.

Actions #5

Updated by Philippe Antoine over 3 years ago

  • Related to Task #3479: libhtp 0.5.33 (4.1.x) added
Actions #6

Updated by Philippe Antoine over 3 years ago

  • Status changed from Assigned to In Review
Actions #7

Updated by Philippe Antoine over 3 years ago

  • Target version changed from TBD to 6.0.0beta1
Actions #8

Updated by Philippe Antoine about 3 years ago

Actions #9

Updated by Victor Julien about 3 years ago

  • Target version changed from 6.0.0beta1 to 6.0.0rc1
Actions #10

Updated by Victor Julien about 3 years ago

  • Target version changed from 6.0.0rc1 to 7.0.0-beta1
Actions #11

Updated by Victor Julien about 3 years ago

Actions #12

Updated by Philippe Antoine about 3 years ago

Actions #13

Updated by Philippe Antoine almost 3 years ago

  • Target version changed from 7.0.0-beta1 to 6.0.1
Actions #14

Updated by Philippe Antoine almost 3 years ago

Actions #15

Updated by Victor Julien almost 3 years ago

  • Target version changed from 6.0.1 to 7.0.0-beta1
Actions #16

Updated by Philippe Antoine almost 3 years ago

Actions #17

Updated by Philippe Antoine almost 3 years ago

https://github.com/OISF/suricata/pull/5599 for 6.0.1

For 7 :
changing the handling in 7 would be good, but I'm not sure it should be optional.

Actions #18

Updated by Philippe Antoine almost 3 years ago

https://github.com/OISF/suricata/pull/5614 merged for 6.0.1

Still work to do for 7

Actions #19

Updated by Philippe Antoine about 2 years ago

Actions #21

Updated by Philippe Antoine about 1 year ago

  • Priority changed from Normal to High
Actions #22

Updated by Victor Julien 11 months ago

  • Target version changed from 7.0.0-beta1 to 7.0.0-rc1
Actions #23

Updated by Philippe Antoine 11 months ago

  • Status changed from In Review to Closed
Actions #24

Updated by Victor Julien 8 months ago

  • Status changed from Closed to In Review
  • Priority changed from High to Normal
  • Target version changed from 7.0.0-rc1 to 8.0.0-beta1

Was accidentally closed. Postponing once more to give rule writers more time to update things on their end.

Actions #25

Updated by Philippe Antoine 6 months ago

  • Target version changed from 8.0.0-beta1 to 7.0.0-rc2
Actions #27

Updated by Philippe Antoine 4 months ago

  • Target version changed from 7.0.0-rc2 to 8.0.0-beta1
Actions #28

Updated by Brandon Murphy 4 months ago

I was testing the v16 fork of this and found a difference between 6.0.9 and v16. I was able to confirm the same behavior in v17 fork.

Current Behavior: When the v17 fork is presented with HTTP/1 Request which contains a double space after the URI and before the Protocol, the extra space is added to the end of the URI.

Expected Behavior: I expected the URI would be normalized and remove any trailing spaces, while the http.uri.raw buffer would contain the space.

alert http $HOME_NET any -> any any (msg:"Test Double Space after URI - alerts on 6.0.9"; flow:established,to_server; http.method; content:"POST"; http.uri; content:"|2e|php"; endswith; sid:1;)
alert http $HOME_NET any -> any any (msg:"Test Double Space after URI - alerts on v17"; flow:established,to_server; http.method; content:"POST"; http.uri; content:"|2e|php|20|"; endswith; sid:2;)
Actions #29

Updated by Philippe Antoine 3 months ago

Thanks @Brandon Murphy for this report.

Would not the solution rather be to consider the URI before the last block of spaces ? (even the raw one)

Otherwise, SCHTPGenerateNormalizedUri needs to add this normalization step (stripping spaces on the right)

Actions #30

Updated by Brandon Murphy 3 months ago

Would not the solution rather be to consider the URI before the last block of spaces ? (even the raw one)

When inconstancies or typos like a double space occur in malicious network traffic, we often use them when creating a rule. The ability to write signatures which can make use of these typos, such as the use of the double space, can provide for good fast_patterns in a rule, along with a low FP rate and the ability to very confidently attribute the traffic to a specific malware family/actor, etc.

I'm not sure where/how the best place to allow that to occur is (if not http.uri.raw). Perhaps http.start is the answer here? Based on https://github.com/OISF/suricata/pull/8869 it appears to have not been selected for "overloading" so that limits our use of http.start

Actions #31

Updated by Philippe Antoine 3 months ago

When inconstancies or typos like a double space occur in malicious network traffic, we often use them when creating a rule.

Indeed

Could http.request_line be used in this case ?

Actions #32

Updated by Philippe Antoine 10 days ago

  • Subject changed from http.protocol parsing inaccuracy to http.protocol parsing inaccuracy : accept spaces in URI
Actions

Also available in: Atom PDF