Use bloomfilter for filemd5
To reduce memory usage, use bloom filters.
There is already code implemented in suricata source
It is very efficient for blacklists.
It might not be efficient for whitelists.
Since it has false positives, it would probably be necessary to do a second level validation lookup from data on disk and it will be more expensive.
Implementing through a different keyword (filemd5bloom?) will help avoiding misuse by users.