Project

General

Profile

Actions

Documentation #3274

open

doc: some inconsistency between http docs keywords description

Added by Peter Manev almost 6 years ago. Updated 11 days ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

It seems the command line help sometimes differs form the online docs in terms of what is sticky buffer.
Example http.uri and http.uri.raw

https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-keywords

and

 /opt/suritest/bin/suricata --list-keywords=http.uri
= http.uri =
Description: sticky buffer to match specifically and only on the normalized HTTP URI buffer
Features: No option,sticky buffer
Documentation: https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-uri-and-http-raw-uri

/opt/suritest/bin/suricata --list-keywords=http.uri.raw
= http.uri.raw =
Description: sticky buffer to match specifically and only on the raw HTTP URI buffer
Features: No option,sticky buffer
Documentation: https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-uri-and-http-raw-uri

/opt/suritest/bin/suricata -V
This is Suricata version 5.0.0-dev (64a789bbf 2019-10-18)

Actions #1

Updated by Andreas Herz over 5 years ago

  • Assignee set to OISF Dev
  • Target version set to 70
Actions #2

Updated by Andreas Herz over 5 years ago

  • Tracker changed from Bug to Documentation
Actions #3

Updated by Victor Julien about 5 years ago

  • Target version changed from 70 to TBD
Actions #4

Updated by Jason Taylor about 1 year ago

  • Assignee changed from OISF Dev to Jason Taylor
Actions #5

Updated by Jason Taylor 13 days ago

I spent some time looking into this and wanted to get some feedback.

I am thinking, to keep the --list-keywords data always up to date, if we could have a script that runs at configure/build time (hooks in via the Makefile?) that would update the keyword descriptions based off the content in the relevant documentation under doc/userguide/rules?

If that part sounds good, the next step I am unsure on what the best way to approach the code insertion modify would be? I did some initial searching and it seems like we could use an additional header file that contains the macro that would get updated with the new documentation snippet (if needed).
The header file of course would then be added as an include to the relevant .c files.

This seems a little intrusive but at the same time, should be a one and done kind of thing that just takes care of itself now. When the docs get updated the associated --list-keyword descriptions would get updated as well at build time.

Thanks in advance for any feedback!

JT

Actions #6

Updated by Jason Ish 11 days ago

I'm not sure I follow why code insertion is needed. The command line help should be correct, or should be made to be correct as its built from what it knows about the code?

Its the documentation that gets out of sync, so isn't it the documentation that needs to be updated?

Maybe I'm thinking the inverse.. Documentation should be generated from the code where applicable. We should not generate code from documentation as the code is the authoritative source on if a keyword is a sticky buffer or not.

Actions #7

Updated by Jason Taylor 11 days ago ยท Edited

Jason Ish wrote in #note-6:

I'm not sure I follow why code insertion is needed. The command line help should be correct, or should be made to be correct as its built from what it knows about the code?

Its the documentation that gets out of sync, so isn't it the documentation that needs to be updated?

Maybe I'm thinking the inverse.. Documentation should be generated from the code where applicable. We should not generate code from documentation as the code is the authoritative source on if a keyword is a sticky buffer or not.

Yeah, I was operating from the perspective that in this case the documentation is updated/correct, where what is in the code is incorrect. As it stands now the documentation is more accurate than what is in the code for the --list-keywords output. If we want the code to be the source of truth, what would be the best way to fix current situation and the documentation more in line with the code (once updated where appropriate)?

JT

Actions #8

Updated by Jason Ish 11 days ago

I thought the code would always be correct. If we look at http.uri, we have a flag for being a sticky buffer:

    /* http.uri sticky buffer */
    sigmatch_table[DETECT_HTTP_URI].name = "http.uri";
    sigmatch_table[DETECT_HTTP_URI].alias = "http.uri.normalized";
    sigmatch_table[DETECT_HTTP_URI].desc = "sticky buffer to match specifically and on
ly on the normalized HTTP URI buffer";
    sigmatch_table[DETECT_HTTP_URI].url = "/rules/http-keywords.html#http-uri-and-http
-uri-raw";
    sigmatch_table[DETECT_HTTP_URI].Setup = DetectHttpUriSetupSticky;
    sigmatch_table[DETECT_HTTP_URI].flags |= SIGMATCH_NOOPT|SIGMATCH_INFO_STICKY_BUFFE
R;

but maybe that is just informational...

But for http.uri and http.uri.raw, aren't the code and the docs in sync with respect to them being sticky buffers? Do we have other examples where they are not in sync?

Actions #9

Updated by Jason Taylor 11 days ago

Jason Ish wrote in #note-8:

I thought the code would always be correct. If we look at http.uri, we have a flag for being a sticky buffer:
[...]
but maybe that is just informational...

But for http.uri and http.uri.raw, aren't the code and the docs in sync with respect to them being sticky buffers? Do we have other examples where they are not in sync?

I'll take another look and see about discrepancies between code docs and rtd docs. One other thing I noticed is the uri anchors/links don't take a user directly to the http keyword (in at least some cases, this is due to the rework of the http keywords page a little while back I think). Which was another aspect to the question of how we could work from a source of truth one way or the other to derive the readthedocs docs or the doc links and such in the source.

Actions #10

Updated by Jason Ish 11 days ago

Sounds like a good start would be a script in CI. It would run the --list-keywords command, gather the output, and verify the anchors exist, etc.

Actions

Also available in: Atom PDF