Project

General

Profile

Actions

Task #4125

open
PA PA

Better fuzzing

Task #4125: Better fuzzing

Added by Philippe Antoine over 5 years ago. Updated 9 months ago.

Status:
In Progress
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:
Hardening

Description

List of things to do :
+ fix bugs
+ visualisation of rust coverage
+ performance inspection
+ coverage inspection
+ run targets on corpus as part of CI
+ more fuzz targets

+ structure aware versus pcap (so as not to fuzz libpcap, but somehow still use pcaps in the seed corpus)
+ differential testing with tcp flow splitting

Current proposal : https://github.com/OISF/suricata/pull/5541


Subtasks 1 (0 open1 closed)

Optimization #6189: fuzz: make sure disabled by default layers such as dnp3 get their fuzz targetClosedPhilippe AntoineActions

Related issues 8 (5 open3 closed)

Related to Suricata - Optimization #3591: fuzz: target with pcap, rules and yamlIn ProgressPhilippe AntoineActions
Related to Suricata - Optimization #3590: fuzz: target for dataset/datarep filesAssignedPhilippe AntoineActions
Related to Suricata - Optimization #3589: fuzz: target for iprep data filesAssignedPhilippe AntoineActions
Related to Suricata - Optimization #3588: fuzz: target for reference.configAssignedPhilippe AntoineActions
Related to Suricata - Optimization #3587: fuzz: target for threshold.configAssignedPhilippe AntoineActions
Blocked by Suricata - Task #7130: rust: dependency "time" fails to build on Rust nightlyClosedVictor JulienActions
Copied to Suricata - Optimization #4364: Ideal integration into oss-fuzzRejectedJeff LucovskyActions
Copied to Suricata - Optimization #4365: Ideal integration into oss-fuzzRejectedShivani BhardwajActions

PA Updated by Philippe Antoine over 5 years ago Actions #1

  • Target version set to 7.0.0-beta1

PA Updated by Philippe Antoine over 5 years ago Actions #2

After merge of https://github.com/OISF/suricata/pull/5557

+ fix more bugs
+ visualisation of rust coverage
+ performance inspection
+ coverage inspection (already improved from 40 to 43% in one day on oss-fuzz, and up to 46% three days after)
+ run targets on corpus as part of CI
+ more fuzz targets

+ structure aware versus pcap (so as not to fuzz libpcap, but somehow still use pcaps in the seed corpus)
+ differential testing with tcp flow splitting

PA Updated by Philippe Antoine over 5 years ago Actions #3

https://github.com/OISF/suricata/pull/5616 :
+ fix some bugs in the targets / improve output
+ visualisation of rust coverage : first required step

PA Updated by Philippe Antoine over 5 years ago Actions #4

improving fuzz target initializaition to prevent dummy bug : https://github.com/OISF/suricata/pull/5672

PA Updated by Philippe Antoine about 5 years ago Actions #5

https://github.com/OISF/suricata/pull/5760 to run targets on public corpus as part of CI

PA Updated by Philippe Antoine about 5 years ago Actions #7

PA Updated by Philippe Antoine about 5 years ago Actions #8

TODO : Fuzz when STREAM_MIDSTREAM...

PA Updated by Philippe Antoine about 5 years ago Actions #9

https://github.com/OISF/suricata/pull/5838 to run targets on corpus as part of CI

So, what is left :
+ fix more bugs like https://github.com/OISF/suricata/pull/5850
+ visualisation of rust coverage cf WIP https://github.com/google/oss-fuzz/pull/4697
+ performance inspection + coverage inspection (50% last time I checked and still rising) (fix bugs first)
+ more fuzz targets (cf other redmine tickets)

+ structure aware versus pcap cf https://github.com/OISF/suricata/pull/5818
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved

VJ Updated by Victor Julien about 5 years ago Actions #10

  • Status changed from Assigned to Closed
  • Label Needs backport to 5.0, Needs backport to 6.0 added

JL Updated by Jeff Lucovsky about 5 years ago Actions #11

JL Updated by Jeff Lucovsky about 5 years ago Actions #12

PA Updated by Philippe Antoine about 5 years ago Actions #13

  • Status changed from Closed to In Review
  • Label deleted (Needs backport to 5.0, Needs backport to 6.0)

https://redmine.openinfosecfoundation.org/issues/4366 is closed
But this parent ticket has more things to do cf #9 as last point

PA Updated by Philippe Antoine about 5 years ago Actions #14

So, what is left :
+ fix more bugs
+ performance inspection + coverage inspection
+ more fuzz targets (cf other redmine tickets)
+ structure aware versus pcap cf https://github.com/OISF/suricata/pull/5967
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved
+ fuzz midstream by default for sigpcap ?

Maybe we should also have specific fuzz target per protocol in addition to the generic ones

PA Updated by Philippe Antoine about 5 years ago Actions #15

PA Updated by Philippe Antoine about 5 years ago Actions #16

PA Updated by Philippe Antoine about 5 years ago Actions #17

PA Updated by Philippe Antoine about 5 years ago Actions #18

PA Updated by Philippe Antoine about 5 years ago Actions #19

PA Updated by Philippe Antoine about 5 years ago Actions #20

In addition to midstream, its, async are interesting options.
We should try to fuzz them all

PA Updated by Philippe Antoine almost 5 years ago Actions #21

So, what is left :
+ fix the bugs
+ performance inspection + coverage inspection + more fuzz targets (cf other redmine tickets)
+ fuzz midstream/async/ips conditionally
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved ?

PA Updated by Philippe Antoine almost 5 years ago Actions #22

Performance inspection :
in addition to https://github.com/OISF/suricata/pull/6188, disabling eve.log is a great way to speed up fuzz_sigpcap_aware (from 5 minutes to 3 minutes and 30 seconds on the corpus)

Another prominent function is __sanitizer_cov_trace_cmp8 called from HashListTableFree

The call to ConfGetBool in DetectMpmInitializeAppMpms seems quite expensive as well
Can be optimized away with #ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION

Another idea is to split the target in 2 : one to only the "one single TCP stream" inputs , and one to do whatever pcaps

PA Updated by Philippe Antoine over 4 years ago Actions #23

So, what is left :
+ fix the bugs
+ clusterfuzzlite to fuzz our own private staging branch on gitlab
+ performance inspection + coverage inspection
+ more fuzz targets (cf other redmine tickets like #3587) need to give a bit more thought
+ fuzz midstream/async/ips conditionally : after some optimization on predef target ? + naming trick as for app layer protocols
+ differential testing with tcp flow splitting cf WIP https://github.com/OISF/suricata/pull/4917 to be improved ? after https://redmine.openinfosecfoundation.org/issues/4858

PA Updated by Philippe Antoine over 4 years ago Actions #24

We should also fuzz LUA support cf detect-engine-dns.c not reachable without it

PA Updated by Philippe Antoine over 4 years ago Actions #25

I should put into Suricata-verify the build corpus functionalities for the relevant fuzz targets

PA Updated by Philippe Antoine about 4 years ago Actions #26

Another idea : shorten timeouts

PA Updated by Philippe Antoine almost 4 years ago Actions #27

  • Target version changed from 7.0.0-beta1 to QA

PA Updated by Philippe Antoine over 3 years ago Actions #28

  • Status changed from In Review to In Progress

PA Updated by Philippe Antoine almost 3 years ago Actions #29

  • Subtask #6189 added

PA Updated by Philippe Antoine almost 2 years ago Actions #30

  • Blocked by Task #7130: rust: dependency "time" fails to build on Rust nightly added

PA Updated by Philippe Antoine over 1 year ago Actions #31

  • Label Hardening added

PA Updated by Philippe Antoine 9 months ago Actions #32

  • Tracker changed from Optimization to Task
  • Subject changed from Ideal integration into oss-fuzz to Better fuzzing
Actions

Also available in: PDF Atom