Task #8532
opensuricata-verify: ensure CI covers all tests
Description
And even all checks
This could be done by
- having an option to output a json file for skipped tests/checks
- have a GitHub action that depends on other actions, that would intersect the json files to see if the intersection is not empty
- ensure CI has enough builds with various options/features to be complete
PA Updated by Philippe Antoine about 1 month ago
Another connex idea : see if skipped tests/checks pass
JI Updated by Jason Ish about 1 month ago
Philippe Antoine wrote:
- ensure CI has enough builds with various options/features to be complete
Which CI? Suricata-Verify CI or Suricata CI?
PA Updated by Philippe Antoine about 1 month ago
Suricata-verify CI as it is the one running different versions of suricata
JI Updated by Jason Ish about 1 month ago ยท Edited
Philippe Antoine wrote in #note-3:
Suricata-verify CI as it is the one running different versions of suricata
Doesn't that counter some of the work going on to lessen the CI load? It also duplicates CI work we already do in Suricata. Could looking into automating commands like the following help:
gh workflow run builds.yml -R OISF/suricata -F SV_BRANCH=pr/3057
See: https://github.com/OISF/suricata-verify/pull/3057#issuecomment-4382765627
PA Updated by Philippe Antoine about 1 month ago
Doesn't that counter some of the work going on to lessen the CI load?
No, I do not meant it to.
Maybe I was not precise enough.
I would like PRs like https://github.com/OISF/suricata-verify/pull/3020 not to be green CI in SV because CI just skips the tests
Could looking into automating commands like the following help
I am not sure since I do not know (cannot prove) if suricata CI exercises all SV tests
This is about automating the work that was done in https://redmine.openinfosecfoundation.org/issues/7461
It also duplicates CI work we already do in Suricata.
Maybe there is deduplication to do
PA Updated by Philippe Antoine about 1 month ago
- Related to Task #7461: suricata-verify: pass all tests added
PA Updated by Philippe Antoine 29 days ago
Also, have some CI job in SV run with ASAN to catch https://github.com/OISF/suricata-verify/pull/3059 and make it not green
PA Updated by Philippe Antoine 29 days ago
- Related to Bug #8500: reputation: useless code leads to buffer underflow added
PA Updated by Philippe Antoine 19 days ago
- Status changed from New to Triaged
PA Updated by Philippe Antoine 1 day ago
- Related to Feature #3082: suricata-verify: option to test 'skipped' tests added