Suricata

While looking at a other issue I found that %2f is in fact normalized. How did you see a problem with this?

%2f decode would be set as a server profile - IDS, apache, iis, etc.

For example, we have disabled %2f decoding for apache, since apache by default doesn't decode %2f, but it can be enabled by setting a conf value in apache conf.

We don't provide an option for users to set this character in the config and that's to be provided.

In your case you might be using a profile that enables decoding %2f - ids, generic, iis6/7/7.5

Have added libhtp customizable features to our conf under ids. If you users want to customize their server profile, they can copy it from the default.ids section

I would've rather preferred all of libhtp config to be shipped in a conf file of it's own, but that means we will have to ship 2 files. Not sure what et has to say about this.

<VJ snip poorly formatted block>

formatting's screwed. Have it under pre-formatted tags.

libhtp:

   default-config:
     personality: IDS
     # Can be specified in kb, mb, gb.  Just a number indicates
     # it's in bytes.
     request-body-limit: 3072
     response-body-limit: 3072
     double-decode-path: no
     double-decode-query: no

     # Configures whether backslash characters are treated as path segment
     # separators. They are not on Unix systems, but are on Windows systems.
     # If this setting is enabled, a path such as "/one\two/three" will be
     # converted to "/one/two/three".  Accepted values - yes, no.
     #path-backslash-separators: yes

     # Configures whether consecutive path segment separators will be
     # compressed. When enabled, a path such as "/one//two" will be normalized
     # to "/one/two". The backslash_separators and decode_separators
     # parameters are used before compression takes place. For example, if
     # backslash_separators and decode_separators are both enabled, the path
     # "/one\\/two\/%5cthree/%2f//four" will be converted to
     # "/one/two/three/four".  Accepted values - yes, no.
     #path-compress-separators: yes

     # This parameter is used to predict how a server will react when control
     # characters are present in a request path, but does not affect path
     # normalization.  Accepted values - none or status_400 */
     #path-control-char-handling: none

     # Controls the UTF-8 treatment of request paths. One option is to only
     # validate path as UTF-8. In this case, the UTF-8 flags will be raised
     # as appropriate, and the path will remain in UTF-8 (if it was UTF-8 in
     # the first place). The other option is to convert a UTF-8 path into a
     # single byte stream using best-fit mapping.  Accepted values - yes, no.
     #path-convert-utf8: yes

     # Configures whether encoded path segment separators will be decoded.
     # Apache does not do this, but IIS does. If enabled, a path such as
     # "/one%2ftwo" will be normalized to "/one/two". If the
     # backslash_separators option is also enabled, encoded backslash
     # characters will be converted too (and subseqently normalized to
     # forward slashes).  Accepted values - yes, no.
     #path-decode-separators: yes

     # Configures whether %u-encoded sequences in path will be decoded. Such
     # sequences will be treated as invalid URL encoding if decoding is not
     # desireable.  Accepted values - yes, no.
     #path-decode-u-encoding: yes

     # Configures how server reacts to invalid encoding in path.  Accepted
     # values - preserve_percent, remove_percent, decode_invalid, status_400
     #path-invalid-encoding-handling: preserve_percent

     # Configures how server reacts to invalid UTF-8 characters in path.
     # This setting will not affect path normalization; it only controls what
     # response status we expect for a request that contains invalid UTF-8
     # characters.  Accepted values - none, status_400.
     #path-invalid-utf8-handling: none

     # Configures how server reacts to encoded NUL bytes. Some servers will
     # terminate path at NUL, while some will respond with 400 or 404. When
     # the termination option is not used, the NUL byte will remain in the
     # path.  Accepted values - none, terminate, status_400, status_404.
     # path-nul-encoded-handling: none

     # Configures how server reacts to raw NUL bytes. Some servers will
     # terminate path at NUL, while some will respond with 400 or 404. When
     # the termination option is not used, the NUL byte will remain in the
     # path.  Accepted values - none, terminate, status_400, status_404.
     path-nul-raw-handling: none

     # Sets the replacement characater that will be used to in the lossy
     # best-fit mapping from Unicode characters into single-byte streams.
     # The question mark is the default replacement character. For example ?
     #set-path-replacement-char: ?

     # Controls what the library does when it encounters an Unicode character
     # where only a single-byte would do (e.g., the %u-encoded characters).
     # Conversion always takes place; this parameter is used to correctly
     # predict the status code used in response. In the future there will
     # probably be an option to convert such characters to UCS-2 or UTF-8.
     # Accepted values - bestfit, status_400 and status_404.
     #set-path-unicode-mapping: bestfit

   server-config:

     - apache:
         address: [192.168.1.0/24, 127.0.0.0/8, "::1"]
         personality: Apache_2_2
         # Can be specified in kb, mb, gb.  Just a number indicates
         # it's in bytes.
         request-body-limit: 4096
         response-body-limit: 4096
         double-decode-path: no
         double-decode-query: no

     - iis7:
         address:
           - 192.168.0.0/24
           - 192.168.10.0/24
         personality: IIS_7_0
         # Can be specified in kb, mb, gb.  Just a number indicates
         # it's in bytes.
         request-body-limit: 4096
         response-body-limit: 4096
         double-decode-path: no
         double-decode-query: no

Including other yamls is supported already, so that can be an option, although I'm not really interested in doing that right now.

Maybe we should just leave all these options out of the default yaml and document them carefully in the use manual. Then we add a link to that in the yaml comments.

Victor Julien wrote:

Including other yamls is supported already, so that can be an option, although I'm not really interested in doing that right now.

Maybe we should just leave all these options out of the default yaml and document them carefully in the use manual. Then we add a link to that in the yaml comments.

This can be done, yeah. Although I'm worried if the wiki would be lost.

Wiki is backed up and we have a script to pull pages from it and convert them into txt. Those are being shipped with the source code. Need to update it though :)

cool. Will add it to the wiki and enable parsing these options in our htp config.

https://github.com/inliniac/suricata/pull/100