Suricata

Some tests use a check.sh script to run "cmp" on 2 files to check that they are the same. This could be converted to an internal check and become part of test.yaml.

Example test: output-tcp-data

Possible YAML syntax for test.yaml:

checks:
  - file-compare:
      filename: tcp-data.log
      expected: expected/tcp-data.log

This will require run.py to parse this kind of test, then do the file comparison internally. It looks like the standard library has a filecmp module which may be useful for this.