Project

General

Profile

Actions

Task #8532

open
PA OD

suricata-verify: ensure CI covers all tests

Task #8532: suricata-verify: ensure CI covers all tests

Added by Philippe Antoine 6 days ago. Updated 4 days ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
Effort:
Difficulty:
Label:

Description

And even all checks

This could be done by
- having an option to output a json file for skipped tests/checks
- have a GitHub action that depends on other actions, that would intersect the json files to see if the intersection is not empty
- ensure CI has enough builds with various options/features to be complete


Related issues 2 (1 open1 closed)

Related to Suricata - Task #7461: suricata-verify: pass all testsClosedPhilippe AntoineActions
Related to Suricata - Bug #8500: reputation: useless code leads to buffer underflowIn ReviewSergey PinaevActions

PA Updated by Philippe Antoine 6 days ago Actions #1

Another connex idea : see if skipped tests/checks pass

JI Updated by Jason Ish 6 days ago Actions #2

Philippe Antoine wrote:

- ensure CI has enough builds with various options/features to be complete

Which CI? Suricata-Verify CI or Suricata CI?

PA Updated by Philippe Antoine 6 days ago Actions #3

Suricata-verify CI as it is the one running different versions of suricata

JI Updated by Jason Ish 6 days ago ยท Edited Actions #4

Philippe Antoine wrote in #note-3:

Suricata-verify CI as it is the one running different versions of suricata

Doesn't that counter some of the work going on to lessen the CI load? It also duplicates CI work we already do in Suricata. Could looking into automating commands like the following help:

gh workflow run builds.yml -R OISF/suricata -F SV_BRANCH=pr/3057

See: https://github.com/OISF/suricata-verify/pull/3057#issuecomment-4382765627

PA Updated by Philippe Antoine 5 days ago Actions #5

Doesn't that counter some of the work going on to lessen the CI load?

No, I do not meant it to.
Maybe I was not precise enough.
I would like PRs like https://github.com/OISF/suricata-verify/pull/3020 not to be green CI in SV because CI just skips the tests

Could looking into automating commands like the following help

I am not sure since I do not know (cannot prove) if suricata CI exercises all SV tests

This is about automating the work that was done in https://redmine.openinfosecfoundation.org/issues/7461

It also duplicates CI work we already do in Suricata.

Maybe there is deduplication to do

PA Updated by Philippe Antoine 5 days ago Actions #6

  • Related to Task #7461: suricata-verify: pass all tests added

PA Updated by Philippe Antoine 4 days ago Actions #7

Also, have some CI job in SV run with ASAN to catch https://github.com/OISF/suricata-verify/pull/3059 and make it not green

PA Updated by Philippe Antoine 4 days ago Actions #8

  • Related to Bug #8500: reputation: useless code leads to buffer underflow added
Actions

Also available in: PDF Atom