Project

General

Profile

Actions

Feature #6290

closed

support case insensitive testing of HTTP header name existence

Added by Brandon Murphy over 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Consider the initial opening comment on https://redmine.openinfosecfoundation.org/issues/5774.

Many rules are written with content negations on HTTP header names such as:

http.header_names; content:!"Accept-"; content:!"Referer|0d 0a|";

However, due to HTTP/2 being lowercase header names these content negations fail to provide the desired effect, as they are case sensitive matches.

Yes, we can add nocase to all of these rules, though as discussed within https://redmine.openinfosecfoundation.org/issues/5774, I'm not sure if that is the best option.

One option I did find when looking into supporting Snort 3 is the http_header_test keyword - https://docs.snort.org/rules/options/payload/http/header_test

# check that the User-Agent field is absent
http_header_test:field user-agent,absent;

as per Snort 3's http_header docs, the header name is case insensitive

Snort 3 also allows users the ability to look for content matches in specific HTTP header fields with the optional field header_name argument. This option is specified with a colon character after http_header, followed by the word "field", and lastly followed by the specific header field name (which is case-insensitive).

This request is to introduce a similar capability within Suricata.

Alternatives considered:
1. http.header_names provides logic to perform the same check while adding the nocase statement to each content. There are currently 10,000+ rules within the ETPRO ruleset that use the http.header_names keyword, some of which have multiple content matches/negations against that buffer that would all need updated. Making such a large number of modifications to existing rules is disruptive to our customer base.

Related Issues:
https://redmine.openinfosecfoundation.org/issues/5775 - This feature request stems from a "random thought" I had when talking through.


Subtasks 1 (0 open1 closed)

Feature #6613: support case insensitive testing of HTTP header name existence (7.0.x backport)ClosedPhilippe AntoineActions

Related issues 2 (2 open0 closed)

Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Feature #5774: Addressing Mixed Case in HTTP Headers Names and HTTP2NewOISF DevActions
Actions #1

Updated by Philippe Antoine about 1 year ago

  • Status changed from New to In Review
  • Assignee changed from OISF Dev to Philippe Antoine
  • Target version changed from TBD to 8.0.0-beta1
Actions #2

Updated by Philippe Antoine about 1 year ago

  • Related to Task #6443: Suricon 2023 brainstorm added
Actions #3

Updated by Brandon Murphy about 1 year ago

from https://redmine.openinfosecfoundation.org/issues/5774
Brandon Murphy wrote in #note-9:

So far, I'm under the impression that for individual HTTP headers https://redmine.openinfosecfoundation.org/issues/5775 seems to be the path forward.

Though this leaves http.header, http.header_names, and http.start needing a solution.

For those buffers, it would be nice to have a solution that will allow for case-insenstive matching of HTTP header names while still allowing for case sensitive header values (excluding http.header_names)

Based on the PR, the proposed solution is to introduce a new transformation header_lowercase

This would work well for http.header and http.start, but would require the following changes to every rule using them.

  1. add the transform after the keyword has been declared
  2. lowercase the content of all header names in any content keyword usages.
  1. 2 represents an issue to programmatically update an existing ruleset, as it assumes all header names can be detected. However due to the various methods of identifying a header name in a content match, this can prove to be a large challenge. Though this challenge is likely to be present regardless of any solution.

I believe a similar approach could be used for http.header_names, though without the complexity of header_lowercase and instead depend on the to_lowercase transformation requested in https://redmine.openinfosecfoundation.org/issues/6439

Actions #4

Updated by Brandon Murphy about 1 year ago

As mentioned during Suricon 2023 brainstorm sessions, I do want to stress that it is very likely that any solution will impact a very large number of rules. Once those rules are published it will be very difficult to "change course" in how we address case differences in HTTP/1 and HTTP/2.

Actions #5

Updated by Philippe Antoine about 1 year ago

  • Related to Feature #5774: Addressing Mixed Case in HTTP Headers Names and HTTP2 added
Actions #6

Updated by Philippe Antoine about 1 year ago

  • Status changed from In Review to Closed

https://github.com/OISF/suricata/pull/9837 was merged

@Brandon Murphy do you want something more ?

Actions #7

Updated by Victor Julien about 1 year ago

  • Status changed from Closed to Resolved
  • Label Needs backport to 7.0 added
Actions #8

Updated by OISF Ticketbot about 1 year ago

  • Subtask #6613 added
Actions #9

Updated by OISF Ticketbot about 1 year ago

  • Label deleted (Needs backport to 7.0)
Actions #10

Updated by Philippe Antoine about 1 year ago

  • Status changed from Resolved to Closed
Actions #11

Updated by Brandon Murphy 12 months ago

Is this eligible for backport into 7?

Actions

Also available in: Atom PDF