Bug #4874: FN when using stream_size with http proto and buffers - Suricata - Open Information Security Foundation

Actions

Copy link

Bug #4874

open

FN when using stream_size with http proto and buffers

Added by Brandon Murphy about 3 years ago. Updated over 1 year ago.

Status:

New

Priority:

Normal

Assignee:

OISF Dev

Target version:

TBD

Affected Versions:

5.0.8, 6.0.4, git master

Effort:

Difficulty:

Label:

Needs Suricata-Verify test

Description

I'm attempting to write a signature for a single HTTP request which occurs as the initial HTTP request after the 3WHS has been established. To accomplish this, I am attempting to use stream_size using `server,<,5;`. In theory this should limit the alert to fall within the server's relative seq number of 4 or less, which during a "normal" 3HWS, should be 1.

In doing this, I have observed an unexpected behavior when using stream_size in conjunction with the http protocol. Consider the following three rules and the attached pcap which contains a single tcp session with a single HTTP POST request.

alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"http proto/buffers -- stream_size < 5"; flow:established,to_server; stream_size:server,<,5; http.method; content:"POST"; http.uri; content:".php"; endswith; sid:1; rev:1;)
alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"http proto/buffers -- stream_size = 237"; flow:established,to_server; stream_size:server,=,237; http.method; content:"POST"; http.uri; content:".php"; endswith; sid:2; rev:1;)
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"tcp proto/no buffers -- stream_size < 5"; flow:established,to_server; stream_size:server,<,5; content:"POST"; startswith; content:".php HTTP"; distance:0; sid:3; rev:1;)

I am expecting that sid:1 and sid:3 to alert. In practice, only sid:2 and sid:3 alert.

Observe the alert timestamp and packet counts included in the eve-log for sid:2 indicates the signature alerts on packet 8 - which is the ACK packet from the client ack'ing the server response (relative seq number is 237). Based on this output, it would appear that Suricata is waiting for response body before alerting, which due to the size of the server response body, the stream_size is now well beyond the initial 3WHS.

In order to alert on the correct packet while enforcing the stream_size to strictly limit the POST request to be the initial request after 3HWS, I was forced to use unbuffered content matches as observed in sid:3, which fires on the correct packet.

sid:2 alert details

  {
    "timestamp": "2019-12-01T17:24:04.618350+0000",
    "flow_id": 990144432969329,
    "pcap_cnt": 8,
    "event_type": "alert",
    "src_ip": "192.168.1.2",
    "src_port": 1038,
    "dest_ip": "57.117.72.90",
    "dest_port": 80,
    "proto": "TCP",
    "community_id": "1:pBMbxC1BNQ1wTGp6aw1g26uN6dw=",
    "tx_id": 0,
    "alert": {
      "action": "allowed",
      "gid": 1,
      "signature_id": 2,
      "rev": 1,
      "signature": "http proto/buffers -- stream_size = 237",
      "category": "",
      "severity": 3
    },
    "http": {
      "hostname": "185.117.72.90",
      "url": "/upload.php",
      "http_user_agent": "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)",
      "http_content_type": "text/html",
      "http_method": "POST",
      "protocol": "HTTP/1.1",
      "status": 200,
      "length": 86,
      "http_request_body_printable": "TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVlciBhZGlwaXNjaW5nIGVsaXQuIEFlbmVhbiBjb21tb2RvIGxpZ3VsYSBlZ2V0IGRvbG9yLiBBZW5lYW4gbWFzc2EuIEN1bSBzb2NpaXMgbmF0b3F1ZSBwZW5hdGlidXMgZXQgbWFnbmlzIGRpcyBwYXJ0dXJpZW50IG1vbg==",
      "http_response_body_printable": "<html>\r\n<header><title>This is title</title></header>\r\n<body>\r\nHello\r\n</body>\r\n</html>",
      "http_request_body": "VEc5eVpXMGdhWEJ6ZFcwZ1pHOXNiM0lnYzJsMElHRnRaWFFzSUdOdmJuTmxZM1JsZEhWbGNpQmhaR2x3YVhOamFXNW5JR1ZzYVhRdUlFRmxibVZoYmlCamIyMXRiMlJ2SUd4cFozVnNZU0JsWjJWMElHUnZiRzl5TGlCQlpXNWxZVzRnYldGemMyRXVJRU4xYlNCemIyTnBhWE1nYm1GMGIzRjFaU0J3Wlc1aGRHbGlkWE1nWlhRZ2JXRm5ibWx6SUdScGN5QndZWEowZFhKcFpXNTBJRzF2Ymc9PQ==",
      "http_response_body": "PGh0bWw+DQo8aGVhZGVyPjx0aXRsZT5UaGlzIGlzIHRpdGxlPC90aXRsZT48L2hlYWRlcj4NCjxib2R5Pg0KSGVsbG8NCjwvYm9keT4NCjwvaHRtbD4=" 
    },
    "app_proto": "http",
    "flow": {
      "pkts_toserver": 4,
      "pkts_toclient": 4,
      "bytes_toserver": 678,
      "bytes_toclient": 459,
      "start": "2019-12-01T17:24:04.595569+0000" 
    },
    "payload": "UE9TVCAvdXBsb2FkLnBocCBIVFRQLzEuMQ0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi94LXd3dy1mb3JtLXVybGVuY29kZWQNClVzZXItQWdlbnQ6IE1vemlsbGEvNC4wIChjb21wYXRpYmxlOyBNU0lFIDguMDsgV2luZG93cyBOVCA1LjE7IFRyaWRlbnQvNC4wKQ0KSG9zdDogMTg1LjExNy43Mi45MA0KQ29udGVudC1MZW5ndGg6IDIyMA0KQ2FjaGUtQ29udHJvbDogbm8tY2FjaGUNCg0KVEc5eVpXMGdhWEJ6ZFcwZ1pHOXNiM0lnYzJsMElHRnRaWFFzSUdOdmJuTmxZM1JsZEhWbGNpQmhaR2x3YVhOamFXNW5JR1ZzYVhRdUlFRmxibVZoYmlCamIyMXRiMlJ2SUd4cFozVnNZU0JsWjJWMElHUnZiRzl5TGlCQlpXNWxZVzRnYldGemMyRXVJRU4xYlNCemIyTnBhWE1nYm1GMGIzRjFaU0J3Wlc1aGRHbGlkWE1nWlhRZ2JXRm5ibWx6SUdScGN5QndZWEowZFhKcFpXNTBJRzF2Ymc9PQ==",
    "payload_printable": "POST /upload.php HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)\r\nHost: 185.117.72.90\r\nContent-Length: 220\r\nCache-Control: no-cache\r\n\r\nTG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVlciBhZGlwaXNjaW5nIGVsaXQuIEFlbmVhbiBjb21tb2RvIGxpZ3VsYSBlZ2V0IGRvbG9yLiBBZW5lYW4gbWFzc2EuIEN1bSBzb2NpaXMgbmF0b3F1ZSBwZW5hdGlidXMgZXQgbWFnbmlzIGRpcyBwYXJ0dXJpZW50IG1vbg==",
    "stream": 1,
    "packet": "AAwp2ta4AsCMpz0CCABFAAAoAxxAAIAGtDrAqAECOXVIWgQOAFAnuDeeC87o9VAQQ4XQXQAAAAAAAAAA",
    "packet_info": {
      "linktype": 1
    }

sid:3 alert details

  {
    "timestamp": "2019-12-01T17:24:04.595921+0000",
    "flow_id": 990144432969329,
    "pcap_cnt": 4,
    "event_type": "alert",
    "src_ip": "192.168.1.2",
    "src_port": 1038,
    "dest_ip": "57.117.72.90",
    "dest_port": 80,
    "proto": "TCP",
    "community_id": "1:pBMbxC1BNQ1wTGp6aw1g26uN6dw=",
    "alert": {
      "action": "allowed",
      "gid": 1,
      "signature_id": 3,
      "rev": 1,
      "signature": "tcp proto/no buffers -- stream_size < 5",
      "category": "",
      "severity": 3
    },
    "flow": {
      "pkts_toserver": 3,
      "pkts_toclient": 1,
      "bytes_toserver": 618,
      "bytes_toclient": 62,
      "start": "2019-12-01T17:24:04.595569+0000" 
    },
    "payload": "UE9TVCAvdXBsb2FkLnBocCBIVFRQLzEuMQ0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi94LXd3dy1mb3JtLXVybGVuY29kZWQNClVzZXItQWdlbnQ6IE1vemlsbGEvNC4wIChjb21wYXRpYmxlOyBNU0lFIDguMDsgV2luZG93cyBOVCA1LjE7IFRyaWRlbnQvNC4wKQ0KSG9zdDogMTg1LjExNy43Mi45MA0KQ29udGVudC1MZW5ndGg6IDIyMA0KQ2FjaGUtQ29udHJvbDogbm8tY2FjaGUNCg0KVEc5eVpXMGdhWEJ6ZFcwZ1pHOXNiM0lnYzJsMElHRnRaWFFzSUdOdmJuTmxZM1JsZEhWbGNpQmhaR2x3YVhOamFXNW5JR1ZzYVhRdUlFRmxibVZoYmlCamIyMXRiMlJ2SUd4cFozVnNZU0JsWjJWMElHUnZiRzl5TGlCQlpXNWxZVzRnYldGemMyRXVJRU4xYlNCemIyTnBhWE1nYm1GMGIzRjFaU0J3Wlc1aGRHbGlkWE1nWlhRZ2JXRm5ibWx6SUdScGN5QndZWEowZFhKcFpXNTBJRzF2Ymc9PQ==",
    "payload_printable": "POST /upload.php HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)\r\nHost: 185.117.72.90\r\nContent-Length: 220\r\nCache-Control: no-cache\r\n\r\nTG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVlciBhZGlwaXNjaW5nIGVsaXQuIEFlbmVhbiBjb21tb2RvIGxpZ3VsYSBlZ2V0IGRvbG9yLiBBZW5lYW4gbWFzc2EuIEN1bSBzb2NpaXMgbmF0b3F1ZSBwZW5hdGlidXMgZXQgbWFnbmlzIGRpcyBwYXJ0dXJpZW50IG1vbg==",
    "stream": 0,
    "packet": "AAwp2ta4AsCMpz0CCABFAAHiAxtAAIAGsoHAqAECOXVIWgQOAFAnuDXkC87oCVAYRHCLawAAUE9TVCAvdXBsb2FkLnBocCBIVFRQLzEuMQ0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi94LXd3dy1mb3JtLXVybGVuY29kZWQNClVzZXItQWdlbnQ6IE1vemlsbGEvNC4wIChjb21wYXRpYmxlOyBNU0lFIDguMDsgV2luZG93cyBOVCA1LjE7IFRyaWRlbnQvNC4wKQ0KSG9zdDogMTg1LjExNy43Mi45MA0KQ29udGVudC1MZW5ndGg6IDIyMA0KQ2FjaGUtQ29udHJvbDogbm8tY2FjaGUNCg0KVEc5eVpXMGdhWEJ6ZFcwZ1pHOXNiM0lnYzJsMElHRnRaWFFzSUdOdmJuTmxZM1JsZEhWbGNpQmhaR2x3YVhOamFXNW5JR1ZzYVhRdUlFRmxibVZoYmlCamIyMXRiMlJ2SUd4cFozVnNZU0JsWjJWMElHUnZiRzl5TGlCQlpXNWxZVzRnYldGemMyRXVJRU4xYlNCemIyTnBhWE1nYm1GMGIzRjFaU0J3Wlc1aGRHbGlkWE1nWlhRZ2JXRm5ibWx6SUdScGN5QndZWEowZFhKcFpXNTBJRzF2Ymc9PQ==",
    "packet_info": {
      "linktype": 1
    }

Files

broke_stream_size_http_anon.pcap (1.4 KB) broke_stream_size_http_anon.pcap

Brandon Murphy, 12/06/2021 12:40 AM

Actions

Copy link

Updated by Victor Julien almost 3 years ago

No solution yet, but here's my analysis:

alert tcp any any -> any any (msg:"tcp proto/no buffers -- stream_size < 5"; flow:established,to_server; stream_size:server,<,5; content:"POST"; startswith; content:".php HTTP"; distance:0; sid:3; rev:1;)

No buffers, no protocol. Matches in packet 4. Matches first data packet, even if there is no reassembled stream yet. This because the sig uses startwith (DEPTH) which forces pkt + stream inspection because Suricata assumes the rule writer intends to match at the start of a PDU when using alert tcp ... content ... depth as many rules are written this way.

alert http any any -> any any (msg:"http proto/buffers -- stream_size < 5"; flow:established,to_server; stream_size:server,<,5; http.method; content:"POST"; http.uri; content:".php"; endswith; sid:1; rev:1;)

No match. See below.

alert http any any -> any any (msg:"http proto/buffers -- stream_size = 237"; flow:established,to_server; stream_size:server,=,237; http.method; content:"POST"; http.uri; content:".php"; endswith; sid:2; rev:1;)

Matches in packet 8. First toserver packet after protocol detection finishes in packet 5. Packet 5 ACKs the data packet 4.
Packet 8 first ACKs the toclient data (Ack=237), then matches the sig: hence sid 1 not matching the < 5 condition. Ack'd data is now 237.

Note that with --simulate-ips it does work, as the protocol detection and stream inspection are immediate then.

Actions

Copy link