Documentation #3274
opendoc: some inconsistency between http docs keywords description
Description
It seems the command line help sometimes differs form the online docs in terms of what is sticky buffer.
Example http.uri and http.uri.raw
https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-keywords
and
/opt/suritest/bin/suricata --list-keywords=http.uri = http.uri = Description: sticky buffer to match specifically and only on the normalized HTTP URI buffer Features: No option,sticky buffer Documentation: https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-uri-and-http-raw-uri /opt/suritest/bin/suricata --list-keywords=http.uri.raw = http.uri.raw = Description: sticky buffer to match specifically and only on the raw HTTP URI buffer Features: No option,sticky buffer Documentation: https://suricata.readthedocs.io/en/latest/rules/http-keywords.html#http-uri-and-http-raw-uri /opt/suritest/bin/suricata -V This is Suricata version 5.0.0-dev (64a789bbf 2019-10-18)
Updated by Andreas Herz over 5 years ago
- Assignee set to OISF Dev
- Target version set to 70
Updated by Andreas Herz over 5 years ago
- Tracker changed from Bug to Documentation
Updated by Victor Julien about 5 years ago
- Target version changed from 70 to TBD
Updated by Jason Taylor about 1 year ago
- Assignee changed from OISF Dev to Jason Taylor
Updated by Jason Taylor 13 days ago
I spent some time looking into this and wanted to get some feedback.
I am thinking, to keep the --list-keywords data always up to date, if we could have a script that runs at configure/build time (hooks in via the Makefile?) that would update the keyword descriptions based off the content in the relevant documentation under doc/userguide/rules?
If that part sounds good, the next step I am unsure on what the best way to approach the code insertion modify would be? I did some initial searching and it seems like we could use an additional header file that contains the macro that would get updated with the new documentation snippet (if needed).
The header file of course would then be added as an include to the relevant .c files.
This seems a little intrusive but at the same time, should be a one and done kind of thing that just takes care of itself now. When the docs get updated the associated --list-keyword descriptions would get updated as well at build time.
Thanks in advance for any feedback!
JT
Updated by Jason Ish 11 days ago
I'm not sure I follow why code insertion is needed. The command line help should be correct, or should be made to be correct as its built from what it knows about the code?
Its the documentation that gets out of sync, so isn't it the documentation that needs to be updated?
Maybe I'm thinking the inverse.. Documentation should be generated from the code where applicable. We should not generate code from documentation as the code is the authoritative source on if a keyword is a sticky buffer or not.
Updated by Jason Taylor 11 days ago ยท Edited
Jason Ish wrote in #note-6:
I'm not sure I follow why code insertion is needed. The command line help should be correct, or should be made to be correct as its built from what it knows about the code?
Its the documentation that gets out of sync, so isn't it the documentation that needs to be updated?
Maybe I'm thinking the inverse.. Documentation should be generated from the code where applicable. We should not generate code from documentation as the code is the authoritative source on if a keyword is a sticky buffer or not.
Yeah, I was operating from the perspective that in this case the documentation is updated/correct, where what is in the code is incorrect. As it stands now the documentation is more accurate than what is in the code for the --list-keywords output. If we want the code to be the source of truth, what would be the best way to fix current situation and the documentation more in line with the code (once updated where appropriate)?
JT
Updated by Jason Ish 11 days ago
I thought the code would always be correct. If we look at http.uri
, we have a flag for being a sticky buffer:
/* http.uri sticky buffer */ sigmatch_table[DETECT_HTTP_URI].name = "http.uri"; sigmatch_table[DETECT_HTTP_URI].alias = "http.uri.normalized"; sigmatch_table[DETECT_HTTP_URI].desc = "sticky buffer to match specifically and on ly on the normalized HTTP URI buffer"; sigmatch_table[DETECT_HTTP_URI].url = "/rules/http-keywords.html#http-uri-and-http -uri-raw"; sigmatch_table[DETECT_HTTP_URI].Setup = DetectHttpUriSetupSticky; sigmatch_table[DETECT_HTTP_URI].flags |= SIGMATCH_NOOPT|SIGMATCH_INFO_STICKY_BUFFE R;
but maybe that is just informational...
But for http.uri and http.uri.raw, aren't the code and the docs in sync with respect to them being sticky buffers? Do we have other examples where they are not in sync?
Updated by Jason Taylor 11 days ago
Jason Ish wrote in #note-8:
I thought the code would always be correct. If we look at
http.uri
, we have a flag for being a sticky buffer:
[...]
but maybe that is just informational...But for http.uri and http.uri.raw, aren't the code and the docs in sync with respect to them being sticky buffers? Do we have other examples where they are not in sync?
I'll take another look and see about discrepancies between code docs and rtd docs. One other thing I noticed is the uri anchors/links don't take a user directly to the http keyword (in at least some cases, this is due to the rework of the http keywords page a little while back I think). Which was another aspect to the question of how we could work from a source of truth one way or the other to derive the readthedocs docs or the doc links and such in the source.