Project

General

Profile

Actions

Bug #1275

closed

ET Rule 2003927 not matchin in suricata

Added by Andreas Herz about 8 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

I have the following rule (from ET) included in my suricata.yaml:

drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Suspicious User-Agent (HTTPTEST) - Seen used by downloaders"; flow:to_server,established; content:"User-Agent|3a| HTTPTEST"; nocase; http_header; threshold: type limit, count 2, track by_src, seconds 300;reference:url,doc.emergingthreats.net/bin/view/Main/2003927; classtype:trojan-activity; sid:2003927; rev:9;)

This rule is detected by snort but not by suricata 2.0.3 when i use the following command:
exec 80<>/dev/tcp/10.0.13.134/80 && ( echo -e 'GET / HTTP/1.0\r\nUser-Agent: Autotest httpRequest\r\nUser-Agent: HTTPTEST\r\n\r\n' >&80; while read -t 10 <&80; do echo $REPLY; done; exec 80<&- 80>&- )

It's working as soon as i remove the first User-Agent. But IMHO the rule should even match with the additional User-Agent in front of the HTTPTEST.
Victor said it could be that it's rejected in an earlier stage, but that means such traffic could go through although i wanted to drop it.


Files

log.pcap.1410875272 (1.59 KB) log.pcap.1410875272 Andreas Herz, 09/16/2014 11:10 AM
Actions #1

Updated by Andreas Herz about 8 years ago

The related pcap file:

Actions #2

Updated by Victor Julien about 8 years ago

  • Status changed from New to Feedback

Please add results of http-events.rules against this traffic.

Actions #3

Updated by Andreas Herz about 8 years ago

As requested i testet it against the http-events.rules, but they don't trigger either.

Actions #4

Updated by Victor Julien about 8 years ago

Interestingly, it seems that the 2 user-agent headers are merged:

09/16/2014-15:47:52.901650 <hostname unknown> [**] / [**] Autotest httpRequest, HTTPTEST [**] 10.0.13.39:37150 -> 10.0.13.134:80

Actions #6

Updated by Andreas Herz almost 8 years ago

So is this intended or something that should be "fixed"?

Actions #7

Updated by Andreas Herz almost 8 years ago

Shall i commit a bug report on github for libhtp or is this already known in upstream libhtp?

Actions #8

Updated by Andreas Herz almost 7 years ago

  • Assignee set to Andreas Herz
  • Target version set to TBD
Actions #9

Updated by Andreas Herz over 6 years ago

Still the same with 3.0 and libhtp 0.5.18 so worth reporting to libhtp upstream?

Actions #10

Updated by Andreas Herz about 4 years ago

  • Assignee changed from Andreas Herz to Anonymous
Actions #11

Updated by Andreas Herz over 3 years ago

  • Assignee set to Community Ticket
Actions #12

Updated by Andreas Herz about 3 years ago

Still valid issue with 5.0

Actions #13

Updated by Victor Julien about 3 years ago

  • Status changed from Feedback to Assigned
  • Assignee changed from Community Ticket to Philippe Antoine

Philippe, could you see what we can do about this?

Actions #14

Updated by Philippe Antoine about 3 years ago

Well, I see this as about Suricata rule keywords http_header and http_raw_header definition

The rule triggers when we use http_raw_header instead of http_header

The rule does not trigger with http_header keyword because there is a normalization step first
That normalization includes concatenating together the different values for a single header (This is done by libhtp).

That is transforming
User-Agent: Autotest httpRequest
User-Agent: HTTPTEST

into
User-Agent: Autotest httpRequest, HTTPTEST

Maybe the best solution would be to have some keywords name and value that we could use with http_header

Actions #15

Updated by Victor Julien almost 3 years ago

I think the normalization is not working in our advantage. I think we should disable the normalization and instead consider this buffer to have 1-N values, instead of just a single one.

This will be quite involved, as it would first require libhtp changes to store this correctly somehow, then in suri we would need to iterate over the 1-N values in detection and logging. The logging (in eve) means we need to change it from scalar to array probably.

Actions #16

Updated by Philippe Antoine almost 3 years ago

then in suri we would need to iterate over the 1-N values in detection

Do you mean that we should then reconstruct N buffers concatenating the one buffered header name with each of the N buffered values ?
Or is there already a motor to do detection across multiple buffers ?

And should we introduce new keywords so as not to break compatibility ?

Actions #17

Updated by Victor Julien almost 3 years ago

At suricon we briefly discussed this issue. I tend to see it as a bug rather than a feature. But I think Philippe you mentioned its documented as a feature?

Actions #18

Updated by Philippe Antoine almost 3 years ago

But I think Philippe you mentioned its documented as a feature?

We have the documentation :
https://suricata.readthedocs.io/en/suricata-5.0.0/rules/http-keywords.html#http-header-and-http-raw-header
which says that the header buffer is normalized, but does not explicitly says that this includes concatenating values

But the concatenation is mentioned explicitly for user agent :
https://suricata.readthedocs.io/en/suricata-4.1.4/rules/http-keywords.html#notes

the values will be concatenated in the http_user_agent buffer, in the order seen from top to bottom, with a comma and space (“, “) between each of them.

What do you think about having http_header_name and http_header_value keywords ?
(Maybe complicated for development, but seems useful for signatures writers)

Actions #19

Updated by Philippe Antoine over 2 years ago

  • Status changed from Assigned to Feedback
Actions #20

Updated by Philippe Antoine about 2 years ago

  • Status changed from Feedback to In Review
  • Assignee changed from Philippe Antoine to Andreas Herz
Actions #21

Updated by Philippe Antoine about 2 years ago

  • Assignee changed from Andreas Herz to Philippe Antoine
Actions #22

Updated by Philippe Antoine almost 2 years ago

  • Target version changed from TBD to 7.0rc1
Actions #23

Updated by Philippe Antoine almost 2 years ago

  • Target version changed from 7.0rc1 to 6.0.1
Actions #24

Updated by Victor Julien almost 2 years ago

  • Status changed from In Review to Closed

Merged pr/5173. I still would like to consider if this behavior is correct, but for now its enough that it is documented.

Actions

Also available in: Atom PDF