Project

General

Profile

Actions

Optimization #3322

open

Use standard CRC32 for hash-like functions

Added by Philippe Antoine almost 3 years ago. Updated over 1 year ago.

Status:
In Review
Priority:
Normal
Target version:
Effort:
low
Difficulty:
Label:

Description

Instead of a custom one (as CRC32 was I think designed for kind of avoiding collisions)

One such function is StringHash cf https://github.com/OISF/suricata/pull/4337


Related issues 1 (1 open0 closed)

Blocked by Bug #4265: QA lab: add possibility to do repeatable replay testsIn ProgressPeter ManevActions
Actions #1

Updated by Victor Julien almost 3 years ago

As we discussed offline, it would be nice to create some kind of benchmarking framework where we could validate such changes. Pure pcap tests may not always give enough insight. For example with the bm optimizations pcap based tests showed no difference, while I think more micro level benchmarks would have shown something.

Actions #2

Updated by Philippe Antoine almost 3 years ago

It would be nice if this benchmarking framework handles caches realistically.

With the example of Boyer-Moore optimizations (one less call to alloc), I am not sure a naive benchmarking would shows much difference as the additional call to alloc would grab repeatedly the same cached memory area, whereas in a real Suricata execution, this would not be the case

Actions #3

Updated by Andreas Herz almost 3 years ago

  • Assignee set to Philippe Antoine
  • Target version set to TBD
Actions #5

Updated by Philippe Antoine almost 3 years ago

I put a first benchmark here
https://github.com/OISF/suricata/pull/4371

Actions #6

Updated by Philippe Antoine over 2 years ago

  • Status changed from New to In Review
Actions #7

Updated by Roland Fischer over 2 years ago

There are a few other hash functions that might be interesting. Google's CityHash comes to mind as it's their "general purpose" hash. SpookyHash/xxhash might be options as well. The stackexchange page lists them as well I think.

Ultimately, it depends on how much time you want to spend on this vs how happy you are with the current hash. Plus, what the usage patterns of the hash are as well as the data to be hashed. ;)

Actions #8

Updated by Victor Julien over 1 year ago

  • Blocked by Bug #4265: QA lab: add possibility to do repeatable replay tests added
Actions #9

Updated by Philippe Antoine over 1 year ago

Actions

Also available in: Atom PDF