Suricata

@Philippe Antoine could you please share your thoughts about this if it makes sense to you?

More questions from me:
1. This pattern (checking for alerts) is not seen in the fuzzer yet, is that intentional?
2. Can fuzzer end up creating some rules that can actually affect/ignore the canary rules leading to no alert?

I guess I miss some context here :
Was there a bug where one rule should have triggered but did not because some other rule was activated ?

But a rule like pass any any -> any any (sid:1;) should indeed prevent all the other rules from triggering, should it not ?

Was there a bug where one rule should have triggered but did not because some other rule was activated ?

This is to ensure that newer changes to the core part of the engine does not cause any changes in the behavior of Suricata. One such recent change is the change in the entire algorithm and data structures around Port Grouping. Although we do have several unit tests to cover many edge cases, we wondered if we still missed any and if fuzzing could help.
There can ofc be a way to do some isolated target but then Victor proposed this idea of checking all kinds of rules and not just specific to port grouping.

But a rule like pass any any -> any any (sid:1;) should indeed prevent all the other rules from triggering, should it not ?

Indeed. Is there a way to tell the fuzzer to skip such kinds of rules? Maybe a seed that only contains alert rules? wdyt?

But a rule like pass any any -> any any (sid:1;) should indeed prevent all the other rules from triggering, should it not ?

Indeed. Is there a way to tell the fuzzer to skip such kinds of rules? Maybe a seed that only contains alert rules? wdyt?

We can have the fuzz target have only alert rules
But will that be enough ?

Like, there is a maximum alerts per packet, so I could hide an alert, by triggering more alerts on the same packet...

If you want to experiment this, you can start from the fuzz_sigpcap_aware target.
The fuzz input is rules + pcap. You should add to the input a new field interpreted as rules, then check if the alerts count are equal for set1 vs set1+set2 on the same pcap
That looks more complete to me than just using a predefined ruleset+pcap