Suricata

Did Suricata log any more info?

I can reproduce this on ArchLinux as well. Even with `-vvvv` a user would only see the mentioned error message from hyperscan. With the amount of commits we might use a git bisect to find out where they broke it and report it upstream.

We were able to narrow it down that the commit https://github.com/intel/hyperscan/commit/277fc400892ba57ad5d9eda9f5bcbc6cc6a1b8ca is the one that introduced the issue. The one before works okay so far. @Dylan Hooker if you would like feel free to confirm that this commit breaks it for you as well and that the commit before https://github.com/intel/hyperscan/commit/5aa4bd565fb1eef6c0ef1b04f6c823e6503bd5b4 still works.

Yep, can confirm:
compiling with https://github.com/intel/hyperscan/commit/277fc400892ba57ad5d9eda9f5bcbc6cc6a1b8ca exhibits the crash on startup
compiling with https://github.com/intel/hyperscan/commit/5aa4bd565fb1eef6c0ef1b04f6c823e6503bd5b4 does not exhibit the crash on startup

It bombs out just after capture threads finish starting. This is the only error logged in verbose mode.
2/3/2023 -- 15:16:23 - <Info> - All AFP capture threads are running.
2/3/2023 -- 15:16:23 - <Error> - [ERRCODE: SC_ERR_FATAL(171)] - Hyperscan returned error -1

I'm investigating ....

The addition of the crc consistency check in master validScratch when called from hs_scan causes the issue.

     /* add quick rose sanity checks by db crc*/
     if (s->db_crc != crc) {
         DEBUG_PRINTF("Improper scratch for current db\n");
         return 0;
     }

The Hyperscan documentation [https://intel.github.io/hyperscan/dev-reference/performance.html?highlight=scratch] recommends each context have it's own scratch space for performance:

A scratch space can be allocated so that it can be used with any one of a number of databases. Each concurrent scan operation (such as a thread) needs its own scratch space.

I manually added the same check to 5aa4bd565fb1eef6c0ef1b04f6c823e6503bd5b4 and removed the early return statement. There were no detected inconsistencies between db_crc and crc

With that check removed, Suricata's unittests and suricata-verify tests execute successfully.

Hyperscan will be updated: https://github.com/intel/hyperscan/issues/350#issuecomment-1455162953

The issue was addressed within hyperscan with https://github.com/intel/hyperscan/commit/f129147a3818e7bdad8a6d5c0e25c3c4ad76fbdb