Suricata

Some keywords are now working. Need to complete with all keywords

One keyword may not be translated :
http.stat_msg has no HTTP2 meaning (besides translating the status code)

http.start does not seem to make sense for HTTP2 in my humble opinion

http.response_body seems to be handled by file_data for HTTP2

There is no such thing as http.response_line in HTTP2

nor http.request_line

There is no http.protocol, or it is always HTTP2

http.request_body should be covered by file_data

Should HTTP2MimicHttp1Request translate headers names ? like Host becomes :authority from HTTP1 to HTTP2

How would we do the http.host normalisation ?

Should we concatenate the values in case there are multiple times the same header (name) in HTTP2 ?

http.cookie calls DetectAppLayerMpmRegister2 with SIG_FLAG_TOCLIENT and HTP_REQUEST_HEADERS, that should rather be HTP_RESPONSE_HEADERS, right ?

https://github.com/OISF/suricata/pull/6087 was merged towards this ticket. It is not complete as some body keywords are missing as mentioned in the PR:

http.request_body and http.response_body, covered by file_data

What remains to be done :

- http.host : do the same normalization... same for http.header. For http.header.raw it is not raw in HTTP2, we need to concatenate key and value. For http.header_names, we can have linefeeds in HTTP2 header names, should we escape them ?
- Concatenate when we get multiple values for one header name cf https://suricata.readthedocs.io/en/suricata-6.0.0/rules/http-keywords.html#id2 example request with 2 Hosts ?
- Make HTTP2MimicHttp1Request translate header names (Host becomes :authority) ?
- http.request_body and http.response_body, covered by file_data. Should we have these specifically ?
- http.request_line and http.response_line do not exist in HTTP2, should we emulate them ? What about http.start ?
- http.protocol and http.stat_msg are implicit, should we emulate them ?

After https://github.com/OISF/suricata/pull/6183
There will be the following questions where we want the opinion of signature writers :
- http.request_body and http.response_body, covered by file_data. Should we have these specifically ?
- http.request_line and http.response_line do not exist in HTTP2, should we emulate them ? What about http.start ?
- http.protocol and http.stat_msg are implicit, should we emulate them ?

There will be the following questions where we want the opinion of signature writers :

- http.request_body and http.response_body, covered by file_data. Should we have these specifically ?
- http.request_line and http.response_line do not exist in HTTP2, should we emulate them ? What about http.start ?
- http.protocol and http.stat_msg are implicit, should we emulate them ?

To retain compatibility of current rules in ET ruleset for example, emulation would likely be necessary of all of these keywords are used fairly heavily

Merge of https://github.com/OISF/suricata/pull/6328

Only emulation remaining

Do you know what you want for 7 rc1 ?

That is do you know what should be done for
- http.request_body and http.response_body, covered by file_data. Should we have these specifically ?
- http.request_line and http.response_line do not exist in HTTP2, should we emulate them ? What about http.start ?
- http.protocol and http.stat_msg are implicit, should we emulate them ?

Some conclusions after a recent conversation about it:

http.protocol would contain HTTP/2
http.request_line would contain: <METHOD> <URI> HTTP/2\r\n
http.request_body (http_client_body) same as file.data toserver
http.response_body (http_server_body) same as file.data toclient

Open question is http.stat_msg. One way could be to use a common dict to map code to message here. E.g. 404 -> "Not found", but this feels very artificial. Another would be to make the buffer "empty". This would mean a negated match against it would work, but no positive matches.

@Brandon Murphy any thoughts on the above?

The current use of http.stat_msg within the ET ruleset could be replaced by using the stat_code instead. Off the top of my head, I would think the stat_msg would be handy is if it doesn't match the normal stat_code. In looking at the ruleset, it seems to have been used as just another way of matching on the stat_code. Looks like a pretty easy effort to remove the usage of the http.stat_msg keyword within the ET ruleset.

This would allow for the buffer being blank for HTTP/2 traffic and in effect, we'd just depreciate the use of it within the ET ruleset unless combined with a http.protocol keyword that would enforce HTTP/1.

Today, we have some rules that want to make use of http.protocol to specifically negate HTTP/2 traffic

zoomequipd why do not you use @alert http1 then ? (I think it would be better for perf as it gets pre filtered first)

I do not know for 6.0.9, but I expected git master to have no alerts on http.protocol; content:"http"; nocase or http.protocol; content:!"http"; nocase with HTTP2 traffic since http.protocol implied HTTP1
Using a constant buffer HTTP/2. will make the first one match (and not the second one).
We could also use an empty buffer, which will make the second one match (and not the first one).

Would you like a linter warning that a rule alert http is in fact alert http1 ? (or alert http2)

Philippe Antoine wrote in #note-29:

Today, we have some rules that want to make use of http.protocol to specifically negate HTTP/2 traffic

zoomequipd why do not you use @alert http1 then ? (I think it would be better for perf as it gets pre filtered first)

Because this was introduced in Suri 6 and we do not currently have a suri 6 specific ruleset to make use of this protocol. Also, it's undocumented (https://suricata.readthedocs.io/en/suricata-6.0.10/rules/intro.html?highlight=protocols#protocol)

I do not know for 6.0.9, but I expected git master to have no alerts on http.protocol; content:"http"; nocase or http.protocol; content:!"http"; nocase with HTTP2 traffic since http.protocol implied HTTP1

You had previously asked if http.protocol should be emulated for HTTP/2. It seems to be for backward compatibility with rules this would be desired.

Using a constant buffer HTTP/2. will make the first one match (and not the second one).

A constant buffer would be my preferred solution, though I'm not sure what the implications are of that as it relates to HTTP/3 or other future versions.

I was looking through some of the previous commits around HTTP2, was http.stat_msg not included in this merge? Maybe it was subsequently removed?

https://github.com/OISF/suricata/pull/5875/commits/5465e0b154ff7ede3e9c423781cf75156d1d1d70

Edit - Is this the implementation of the "map" that Victor mentioned above or it just empty for HTTP/2?

Brandon Murphy wrote in #note-33:

I was looking through some of the previous commits around HTTP2, was http.stat_msg not included in this merge? Maybe it was subsequently removed?

https://github.com/OISF/suricata/pull/5875/commits/5465e0b154ff7ede3e9c423781cf75156d1d1d70

This was a confusion from me at the time between stat-code and stat-msg :-/

Edit - Is this the implementation of the "map" that Victor mentioned above or it just empty for HTTP/2?

In the current PR, I put an empty field for HTTP/2
What do you think about this ?

Because this was introduced in Suri 6 and we do not currently have a suri 6 specific ruleset to make use of this protocol. Also, it's undocumented (https://suricata.readthedocs.io/en/suricata-6.0.10/rules/intro.html?highlight=protocols#protocol)

Indeed, nice catch.
So, I created https://redmine.openinfosecfoundation.org/issues/5962

About this, could a keyword to indicate the minimum suricata version for a rule be helpful ?

You had previously asked if http.protocol should be emulated for HTTP/2. It seems to be for backward compatibility with rules this would be desired.

Sure, but I propose to get master branch to have the correct behavior, then backport/fix master-6

Philippe Antoine wrote in #note-35:

In the current PR, I put an empty field for HTTP/2
What do you think about this ?

I'd prefer it to be HTTP/2 to mimic the behavior as it its with HTTP/1

About this, could a keyword to indicate the minimum suricata version for a rule be helpful ?

Yes, this would be really interesting! Perhaps the engine could throw a warning on and not load the rule if the version is too low?
Would you like me to make a feature request?

The empty buffer is http_stat_msg for HTTP2

http.protocol is filled with HTTP/2

Yes, this would be really interesting! Perhaps the engine could throw a warning on and not load the rule if the version is too low?

Would you like me to make a feature request?

Please do so and assign it to me, but check if there exists one already ;-)

Philippe Antoine wrote in #note-37:

The empty buffer is http_stat_msg for HTTP2

Ahhh, ok that sounds fine. I wonder if it would be worth adding an HTTP/2 Nuances section on each keyword that behaves a bit differently for HTTP/2.

http.protocol is filled with HTTP/2

Perfect.

Yes, this would be really interesting! Perhaps the engine could throw a warning on and not load the rule if the version is too low?

Would you like me to make a feature request?

Please do so and assign it to me, but check if there exists one already ;-)

Will do!

https://github.com/OISF/suricata/pull/8687 by the way