Bug #6959
closedimprove handling of content encoding: gzip but request_body not actually compressed
Description
Looking for a little help here.
Context¶
We have a rule (simplified here) designed to alert on traffic contained within 2038629_single_fn_synth.pcap (attached)
alert http $HOME_NET any -> $EXTERNAL_NET 8100 (msg:"ET MOBILE_MALWARE Android Dropper Checkin Activity (POST)"; flow:established,to_server; http.request_body; content:"|22|sdk_version|22 3a|"; content:"|22|app_package|22 3a 22|"; distance:0; content:"|22|app_version|22 3a 22|"; distance:0; content:"|22|device_id|22 3a 22|"; distance:0; classtype:trojan-activity; sid:2038629; rev:1;)
The pcap contains the following HTTP transaction, please note the Content-Encoding: gzip
header, though the request body is not actually compressed.
POST /api/sdk/init2 HTTP/1.1 Content-Encoding: gzip Charset: UTF-8 Accept-Encoding: gzip,deflate Content-Type: application/json User-Agent: Dalvik/2.1.0 (Linux; U; Android 9; Moto G (7) Build/OPPS28.65-37-7-11) Host: sdk.xinxiansk.com:8100 Connection: Keep-Alive Content-Length: 106 {"sdk_version":11026,"app_package":"com.xcm.huasheng","app_version":"1.7.4","device_id":"XXXX"}
Current Behavior¶
This signature fires using 6.0.18, but does not fire on 7.0.4. This discrepancy was found while testing the Suricata 7.0 Emerging Threats ruleset.
Analysis¶
Upon examination, with 7.0.4, it appears the condition of the Content-Encoding: gzip
header is present while the content is not compressed produces an unexpected population of the http.request_body buffer.
Using the lua logging module to log the contents of the http.request_body buffer the following rule which alerts on 7.0.4 was produced.
alert http $HOME_NET any -> $EXTERNAL_NET 8100 (msg:"TEST"; flow:established,to_server; http.request_body; content:"|e4 1c c4 b5 8e 6b 79 63 83 75 8d 6d 61 83 75 8d|"; startswith; classtype:trojan-activity; sid:2; rev:1;)
Suricata 7.0.4 does produce the "GZIP_DECOMPRESSION_FAILED" anomaly event while 6.0.18 does not.
Outstanding Questions¶
- Is the observed difference in behavior between 6.0.18 and 7.0.4 expected?
- From where is the data within http.request_body; coming?
Desired Behavior¶
It would be my preference that the engine behave as it does in 6.0.18 as I currently have no method of writing the signature for 7.0.4 given the buffer is populated with data from an unknown source.
Files
Updated by Victor Julien 8 months ago
- Status changed from New to Assigned
- Assignee changed from OISF Dev to Philippe Antoine
@Philippe Antoine can you check this?
Updated by Philippe Antoine 8 months ago
- Status changed from Assigned to In Review
- Target version changed from TBD to 8.0.0-beta1
Thanks for the report Brandon
Fix is in https://github.com/OISF/libhtp/pull/420
Updated by Victor Julien 8 months ago
- Related to Task #6965: libhtp 0.5.48 added
Updated by Victor Julien 8 months ago
- Label Needs backport to 7.0 added
Fix is in libhtp, but adding backport ticket for the Changelog update.
Updated by Victor Julien 8 months ago
- Status changed from In Review to Closed
Updated by Victor Julien 8 months ago
- Subject changed from behavior discrepancy between 6.0.18 and 7.0.4 when encountering Content-Encoding: gzip but request_body not actually compressed to improve handling of content encoding: gzip but request_body not actually compressed