Suricata

Conversely, the http.url field is only matching up until the first space resulting in something like:

"http":{"url":"/uid=0(root)"}

which leads me to believe those fields are created by splitting the line by spaces.

I think uri's are not supposed to have spaces, but I think it would be good to address this anyway.

Thanks Chris.
Indeed, Uris are not supposed to have spaces, but the protocol field is even less supposed to have spaces.
So I think we can take the last space in the request line as the uri end, instead of the second one.

https://github.com/OISF/suricata/pull/5599 for 6.0.1

For 7 :
changing the handling in 7 would be good, but I'm not sure it should be optional.

https://github.com/OISF/suricata/pull/5614 merged for 6.0.1

Still work to do for 7

https://github.com/OISF/suricata/pull/6884 is latest PR to review

https://github.com/OISF/suricata/pull/8509 currently

Thanks @Brandon Murphy for this report.

Would not the solution rather be to consider the URI before the last block of spaces ? (even the raw one)

Otherwise, SCHTPGenerateNormalizedUri needs to add this normalization step (stripping spaces on the right)

Would not the solution rather be to consider the URI before the last block of spaces ? (even the raw one)

When inconstancies or typos like a double space occur in malicious network traffic, we often use them when creating a rule. The ability to write signatures which can make use of these typos, such as the use of the double space, can provide for good fast_patterns in a rule, along with a low FP rate and the ability to very confidently attribute the traffic to a specific malware family/actor, etc.

I'm not sure where/how the best place to allow that to occur is (if not http.uri.raw). Perhaps http.start is the answer here? Based on https://github.com/OISF/suricata/pull/8869 it appears to have not been selected for "overloading" so that limits our use of http.start

When inconstancies or typos like a double space occur in malicious network traffic, we often use them when creating a rule.

Indeed

Could http.request_line be used in this case ?

Sorry, it looks like I dropped the ball on this one. I've "watched" this ticket to make sure that doesn't happen again.

Could http.request_line be used in this case ?

I guess if this is the only solution, then ok. The issue is that it's uncommon usage of the buffer and lacks normalization of the uri, etc. It would pretty much make http.request_line the new http.uri.raw in cases where a double space is present, which will be an edge case often forgotten about and hard to teach people new to the engine. At the very least, if this is selected as the desired option, would it be possible to add a section to the documents on this behavior?

I feel like anyone looking at network traffic can clearly see that the URI has a space in it and that the protocol is at the end.

Would not the solution rather be to consider the URI before the last block of spaces ? (even the raw one)

RFC says this

   A request-line begins with a method token, followed by a single space
   (SP), the request-target, another single space (SP), the protocol
   version, and ends with CRLF.

     request-line   = method SP request-target SP HTTP-version CRLF

What do you think about considering the URI before the last space? (not the last "block of spaces"). This conforms with format of the request-line as per the RFC.

given a request line of GET /foo.php HTTP/1.1

http.uri = /foo.php (no trailing space, gets normalized out)
http.uri.raw = /foo.php (trailing space)
http.request_line = GET /foo.php HTTP/1.1 (contains the double space)

More or less the method would be anything to the "left" of the first space, the version anything "right" of the last space and anything in between those two is the URI?

https://forum.suricata.io/t/http-protocol-error-field-parsing/4150/2

Issue mentioned here

So, SCHTPGenerateNormalizedUri needs to add this normalization step (stripping spaces on the right), is it correct @Brandon Murphy ?

I won't pretend to be a developer and tell you what function needs updating, I defer to you on that! :-)

Given the behavior mentioned in #note-28, I'm honestly surprised trailing spaces were not already normalized out, but I sure don't know enough to read through libhtp and suricata code to figure it out.