Task #8532
opensuricata-verify: ensure CI covers all tests
Description
And even all checks
This could be done by
- having an option to output a json file for skipped tests/checks
- have a GitHub action that depends on other actions, that would intersect the json files to see if the intersection is not empty
- ensure CI has enough builds with various options/features to be complete
PA Updated by Philippe Antoine 6 days ago
Another connex idea : see if skipped tests/checks pass
PA Updated by Philippe Antoine 6 days ago
Suricata-verify CI as it is the one running different versions of suricata
JI Updated by Jason Ish 6 days ago ยท Edited
Philippe Antoine wrote in #note-3:
Suricata-verify CI as it is the one running different versions of suricata
Doesn't that counter some of the work going on to lessen the CI load? It also duplicates CI work we already do in Suricata. Could looking into automating commands like the following help:
gh workflow run builds.yml -R OISF/suricata -F SV_BRANCH=pr/3057
See: https://github.com/OISF/suricata-verify/pull/3057#issuecomment-4382765627
PA Updated by Philippe Antoine 5 days ago
Doesn't that counter some of the work going on to lessen the CI load?
No, I do not meant it to.
Maybe I was not precise enough.
I would like PRs like https://github.com/OISF/suricata-verify/pull/3020 not to be green CI in SV because CI just skips the tests
Could looking into automating commands like the following help
I am not sure since I do not know (cannot prove) if suricata CI exercises all SV tests
This is about automating the work that was done in https://redmine.openinfosecfoundation.org/issues/7461
It also duplicates CI work we already do in Suricata.
Maybe there is deduplication to do
PA Updated by Philippe Antoine 5 days ago
- Related to Task #7461: suricata-verify: pass all tests added
PA Updated by Philippe Antoine 4 days ago
Also, have some CI job in SV run with ASAN to catch https://github.com/OISF/suricata-verify/pull/3059 and make it not green
PA Updated by Philippe Antoine 4 days ago
- Related to Bug #8500: reputation: useless code leads to buffer underflow added