Optimization #4125
openIdeal integration into oss-fuzz
Description
List of things to do :
+ fix bugs
+ visualisation of rust coverage
+ performance inspection
+ coverage inspection
+ run targets on corpus as part of CI
+ more fuzz targets
+ structure aware versus pcap (so as not to fuzz libpcap, but somehow still use pcaps in the seed corpus)
+ differential testing with tcp flow splitting
Current proposal : https://github.com/OISF/suricata/pull/5541
Updated by Philippe Antoine about 4 years ago
After merge of https://github.com/OISF/suricata/pull/5557
+ fix more bugs
+ visualisation of rust coverage
+ performance inspection
+ coverage inspection (already improved from 40 to 43% in one day on oss-fuzz, and up to 46% three days after)
+ run targets on corpus as part of CI
+ more fuzz targets
+ structure aware versus pcap (so as not to fuzz libpcap, but somehow still use pcaps in the seed corpus)
+ differential testing with tcp flow splitting
Updated by Philippe Antoine almost 4 years ago
https://github.com/OISF/suricata/pull/5616 :
+ fix some bugs in the targets / improve output
+ visualisation of rust coverage : first required step
Updated by Philippe Antoine almost 4 years ago
improving fuzz target initializaition to prevent dummy bug : https://github.com/OISF/suricata/pull/5672
Updated by Philippe Antoine almost 4 years ago
https://github.com/OISF/suricata/pull/5760 to run targets on public corpus as part of CI
Updated by Philippe Antoine almost 4 years ago
Fixing fuzz targets https://github.com/OISF/suricata/pull/5791
Updated by Philippe Antoine almost 4 years ago
Fixing fuzz target with https://github.com/OISF/suricata/pull/5796
Updated by Philippe Antoine almost 4 years ago
TODO : Fuzz when STREAM_MIDSTREAM...
Updated by Philippe Antoine almost 4 years ago
https://github.com/OISF/suricata/pull/5838 to run targets on corpus as part of CI
So, what is left :
+ fix more bugs like https://github.com/OISF/suricata/pull/5850
+ visualisation of rust coverage cf WIP https://github.com/google/oss-fuzz/pull/4697
+ performance inspection + coverage inspection (50% last time I checked and still rising) (fix bugs first)
+ more fuzz targets (cf other redmine tickets)
+ structure aware versus pcap cf https://github.com/OISF/suricata/pull/5818
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved
Updated by Victor Julien over 3 years ago
- Status changed from Assigned to Closed
- Label Needs backport to 5.0, Needs backport to 6.0 added
Updated by Jeff Lucovsky over 3 years ago
- Copied to Optimization #4364: Ideal integration into oss-fuzz added
Updated by Jeff Lucovsky over 3 years ago
- Copied to Optimization #4365: Ideal integration into oss-fuzz added
Updated by Philippe Antoine over 3 years ago
- Status changed from Closed to In Review
- Label deleted (
Needs backport to 5.0, Needs backport to 6.0)
https://redmine.openinfosecfoundation.org/issues/4366 is closed
But this parent ticket has more things to do cf #9 as last point
Updated by Philippe Antoine over 3 years ago
So, what is left :
+ fix more bugs
+ performance inspection + coverage inspection
+ more fuzz targets (cf other redmine tickets)
+ structure aware versus pcap cf https://github.com/OISF/suricata/pull/5967
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved
+ fuzz midstream by default for sigpcap ?
Maybe we should also have specific fuzz target per protocol in addition to the generic ones
Updated by Philippe Antoine over 3 years ago
- Related to Optimization #3591: fuzz: target with pcap, rules and yaml added
Updated by Philippe Antoine over 3 years ago
- Related to Optimization #3590: fuzz: target for dataset/datarep files added
Updated by Philippe Antoine over 3 years ago
- Related to Optimization #3589: fuzz: target for iprep data files added
Updated by Philippe Antoine over 3 years ago
- Related to Optimization #3588: fuzz: target for reference.config added
Updated by Philippe Antoine over 3 years ago
- Related to Optimization #3587: fuzz: target for threshold.config added
Updated by Philippe Antoine over 3 years ago
In addition to midstream, its, async are interesting options.
We should try to fuzz them all
Updated by Philippe Antoine over 3 years ago
So, what is left :
+ fix the bugs
+ performance inspection + coverage inspection + more fuzz targets (cf other redmine tickets)
+ fuzz midstream/async/ips conditionally
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved ?
Updated by Philippe Antoine over 3 years ago
Performance inspection :
in addition to https://github.com/OISF/suricata/pull/6188, disabling eve.log is a great way to speed up fuzz_sigpcap_aware (from 5 minutes to 3 minutes and 30 seconds on the corpus)
Another prominent function is __sanitizer_cov_trace_cmp8
called from HashListTableFree
The call to ConfGetBool
in DetectMpmInitializeAppMpms
seems quite expensive as well
Can be optimized away with #ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
Another idea is to split the target in 2 : one to only the "one single TCP stream" inputs , and one to do whatever pcaps
Updated by Philippe Antoine almost 3 years ago
So, what is left :
+ fix the bugs
+ clusterfuzzlite to fuzz our own private staging branch on gitlab
+ performance inspection + coverage inspection
+ more fuzz targets (cf other redmine tickets like #3587) need to give a bit more thought
+ fuzz midstream/async/ips conditionally : after some optimization on predef target ? + naming trick as for app layer protocols
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved ? after https://redmine.openinfosecfoundation.org/issues/4858
Updated by Philippe Antoine almost 3 years ago
We should also fuzz LUA support cf detect-engine-dns.c not reachable without it
Updated by Philippe Antoine almost 3 years ago
I should put into Suricata-verify the build corpus functionalities for the relevant fuzz targets
Updated by Philippe Antoine over 2 years ago
- Target version changed from 7.0.0-beta1 to QA
Updated by Philippe Antoine almost 2 years ago
- Status changed from In Review to In Progress
Updated by Philippe Antoine 4 months ago
- Blocked by Task #7130: rust: dependency "time" fails to build on Rust nightly added