Bug #7754
openhttp.host and http.host.raw contain the same Host header value twice, with a delimiter
Description
I ran into some strange behaviour when drafting a rule to detect RFC non-compliant characters within the HTTP host header field in which the value of the header appears to be entered into the buffer twice, delimited by ', ', is this intended behaviour?
This rule was firing many times (500k+ hits) across the Emerging Threats QA session today which baffled all of us:
alert http any any > [$HOME_NET,$HTTP_SERVERS] any (msg:"ET HUNTING Non-RFC Compliant HTTP Host Header Observed"; flow:established,to_server; http.host; pcre:"/^.*?[^A-Za-z0-9\\.\[\]:%]/"; classtype:bad-unknown; sid:54000000; rev:1;)
When investigating the PCAPs associated with the alerts, nothing stood out to us. We looked into the hexdumps of those HTTP requests and we verified that the PCAPs were not corrupt in some strange way.
From there we dumped the buffer directly from Suricata-7.0.3 and while I can't share actual domains or PCAPs that we observed in this ticket, I can share that the content structure of the buffer (hexdump) is the same as below.
http.host;
00000000 74 65 73 74 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 2c |test.google.com,|
00000010 20 74 65 73 74 2e 67 6f 6f 67 6c 65 2e 63 6f 6d | test.google.com|
I thought this was an error so I created the following rule:
alert http any any -> [$HOME_NET,$HTTP_SERVERS] any (msg:"ET HUNTING Non-RFC Compliant HTTP Host Header Observed"; flow:established,to_server; http.host; content:"|2c 20|"; classtype:bad-unknown; sid:54000000; rev:1;)
And sure enough, writing a rule to specifically detect the delimiter observed in the dumped buffer gave us alerts.
Updated by James Emery-Callcott 2 days ago
Re-posting the same signature in a code block so certain characters aren't interpreted as formatting efforts.
alert http any any -> [$HOME_NET,$HTTP_SERVERS] any (msg:"ET HUNTING Non-RFC Compliant HTTP Host Header Observed"; flow:established,to_server; http.host; pcre:"/^.*?[^A-Za-z0-9\-\.\[\]:%]/"; classtype:bad-unknown; sid:54000000; rev:1;)
Updated by James Emery-Callcott 2 days ago
Came back to this with fresh eyes and I think I've identified the cause however, it's potentially still a bug.
It seems this behaviour is caused by the presence of multiple 'Host' headers, meaning Suricata will take all values and add them into a list which is never split to evaluate each instance separately. So in this scenario:
GET /reallyabug?unsure.html HTTP/1.1
Accept: */*
Host: test.google.com
Accept-Language: en-us
Host: test.google.com
Accept-Encoding: gzip, deflate
Host: test.google.com
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
Suricata will end up with a buffer of..
00000000 74 65 73 74 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 2c |test.google.com,|
00000010 20 74 65 73 74 2e 67 6f 6f 67 6c 65 2e 63 6f 6d | test.google.com|
00000020 2c 20 74 65 73 74 2e 67 6f 6f 67 6c 65 2e 63 6f |, test.google.co|
00000030 6d |m|
Suricata does not split this buffer on ', ' to process each value separately and instead concats each value to the buffer, making it appear as though this was the HTTP request:
GET /reallyabug?unsure.html HTTP/1.1
Accept: */*
Host: test.google.com, test.google.com, test.google.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
It still seems like a bug to me albeit different from the initial report I created.
Updated by James Emery-Callcott 2 days ago
My concerns here is that this could now lead to bypassing many existing signatures with ease.
If a signature includes something such as the following:
http.host; content:"test.google.com"; bsize:<20;
You can simply add another HTTP host header filled with garbage to increase the buffer size, causing the bsize check to fail and the signature to no longer alert.
Updated by James Emery-Callcott 2 days ago
I've just tested http.accept; for this same logic and it seems that buffer is affected too. I suspect all http sticky buffers are affected by this.
Updated by Victor Julien 1 day ago
This behavior is documented in https://docs.suricata.io/en/latest/rules/http-keywords.html#duplicate-header-names
Related doc ticket was #6781.
Updated by Victor Julien 1 day ago
- Related to Documentation #6781: http: document duplicate headers concatenation handling added
Updated by Victor Julien 1 day ago
- Related to Feature #6925: multi-buffer support for HTTP cookies added
Updated by James Emery-Callcott 1 day ago
Does this mean there is no plan to change this behaviour and that it is intended?