Support #3252
closedMatching a long list of tls.fingerprint fields is extremly CPU intensive
Description
The suricata-update script comes with some useful rule sources, like the sslbl.abuse.ch.
It is basically around 3000 rules with malicious certificate's fingerprints.
This list alone, when combined with the ET Pro ruleset makes the CPU usage on my sensor go from 6% to 60% and even 100% on as little as 400Mbit/sec of traffic.
This sensor is way oversized - Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz x 2 so 24 cores tuned to perfection.
Removing just those 3000 rules dropped the CPU usage to as little as 6%. I repeated the test thrice (just so I could say thrice, because I never get to say thrice).
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,224.0.0.0/4...]"
REALLY_EXTERNAL_NET: "!$HOME_NET"
EXTERNAL_NET: "any"
(yes, I should have rewritten those rules to point to REALLY_EXTERNAL_NET I guess).
Feel free to tell me the engine works like designed and it's just abused here, and the intelligence framework / datasets should be used instead. In this case let's move it to suricata-update so people don't experience problems by default.
This is Suricata version 4.1.5 RELEASE
38d55ce367216ce4c8347f559bc3b153faafae5/; sid:902200862; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"48:25:f4:a2:cb:22:4d:11:74:be:a7:10:04:35:6b:94:3e:42:a2:a4"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/4825f4a2cb224d1174bea71004356b943e42a2a4/; sid:902201062; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"5f:cb:5b:41:8f:77:9a:54:2b:71:48:f2:dd:ea:21:14:95:78:77:33"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/5fcb5b418f779a542b7148f2ddea211495787733/; sid:902201621; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"76:c7:c0:90:dc:32:3f:56:e2:c0:31:11:ca:92:ae:67:ef:a5:8d:b0"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/76c7c090dc323f56e2c03111ca92ae67efa58db0/; sid:902201501; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"79:67:bb:dd:e9:c1:17:46:8d:26:cd:de:db:20:e2:1c:46:63:bd:d7"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/7967bbdde9c117468d26cddedb20e21c4663bdd7/; sid:902200013; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"7f:48:d4:aa:cf:79:49:e3:de:64:6c:61:0b:9c:59:79:c6:8e:c5:2f"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/7f48d4aacf7949e3de646c610b9c5979c68ec52f/; sid:902201002; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"7f:cd:2c:56:08:47:7d:34:c2:a3:9e:0a:74:3a:20:52:dc:de:94:d1"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/7fcd2c5608477d34c2a39e0a743a2052dcde94d1/; sid:902201260; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"82:c0:a9:7f:05:88:93:a7:7f:8a:2a:27:bb:75:b5:fb:7a:d2:30:ac"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/82c0a97f058893a77f8a2a27bb75b5fb7ad230ac/; sid:902201634; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"8f:7e:4e:31:ce:31:6e:3f:ab:9b:a5:34:6c:f4:2e:bb:0f:ed:2d:85"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/8f7e4e31ce316e3fab9ba5346cf42ebb0fed2d85/; sid:902200948; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"a3:93:d2:01:ba:27:f5:5b:3c:d9:86:15:1d:02:f8:68:15:97:60:2c"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/a393d201ba27f55b3cd986151d02f8681597602c/; sid:902201190; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"ac:99:29:98:8c:ab:80:0a:65:3b:01:12:ba:31:6c:47:25:d1:9f:c3"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/ac9929988cab800a653b0112ba316c4725d19fc3/; sid:902201421; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"b6:05:c6:66:e1:9d:5d:bc:06:f5:73:a4:1f:35:94:d1:c9:31:fa:f1"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/b605c666e19d5dbc06f573a41f3594d1c931faf1/; sid:902201636; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"e1:d3:8d:99:d7:b3:d1:9d:d3:c4:0e:8d:a7:ec:d4:d8:b8:5b:67:5e"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/e1d38d99d7b3d19dd3c40e8da7ecd4d8b85b675e/; sid:902201570; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS C&C)"; tls.fingerprint:"fa:c2:3c:75:88:97:f5:90:24:56:6a:fd:da:8d:9e:6c:98:bb:2d:60"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/fac23c758897f59024566afdda8d9e6c98bb2d60/; sid:902201201; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (ZeuS MITM)"; tls.fingerprint:"b6:d7:85:2a:e1:ca:32:5f:77:28:d4:64:12:44:8b:01:41:94:0b:c9"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/b6d7852ae1ca325f7728d46412448b0141940bc9/; sid:902200148; rev:1;)
+alert tls $EXTERNAL_NET any -> $HOME_NET any (msg:"SSLBL: Malicious SSL certificate detected (Zloader C&C)"; tls.fingerprint:"d8:27:5a:0d:7a:e3:27:68:79:7e:c2:cc:f1:c0:fc:2f:f5:98:a1:ae"; reference:url, sslbl.abuse.ch/ssl-certificates/sha1/d8275a0d7ae32768797ec2ccf1c0fc2ff598a1ae/; sid:902202065; rev:1;)
Updated by Andreas Herz about 5 years ago
- Tracker changed from Bug to Support
- Assignee set to Community Ticket
- Target version set to 70
Updated by Victor Julien over 4 years ago
- Status changed from New to Closed
- Assignee deleted (
Community Ticket)
Use tls.cert_fingerprint; dataset:isset,tls-fingerprints;
you should be able to get much better perf.
But even w/o datasets tls.cert_fingerprint
will outperform the legacy tls.fingerprint
.
Updated by Victor Julien over 2 years ago
- Related to Bug #4581: Excessive qsort/msort time when large number of rules using tls.fingerprint added