Suricata

So this is a normal ruleset like ET used with a large HOME_NET? Are you able to share your HOME_NET setting, or create a fake one that has the same effect?

To make it easy to repo I switched to just the full et pro rule set and nothing else. Which outputs

45259 signatures processed. 1225 are IP-only rules, 17090 are inspecting packet payload, 31407 inspect application layer, 0 are decoder event only

So running with 
time /nids/suricata/bin/suricata -c /nids/suricata/etc/suricata.yaml --af-packet -v

Our original ip list:
real    29m15.512s
user    28m49.707s
sys     0m25.756s

Then I sorted the list with: sort -t/ -k2 -n -r
real    11m20.801s
user    11m18.115s
sys     0m2.682s

So sorting helped a lot. I tried to make a sorted fake list, which isn't quite as slow, but still almost 8m.

real    7m58.549s
user    7m56.561s
sys     0m1.986s

"[1111:2222:3333:7fff::a:1/128,1111:2222:3333:7eff::a:1/128,1212:2222:3333:7fff::a:1/128,1212:2222:3333:7eff::a:1/128,1313:2222:3333:ff::a:2/128,1313:2222:3333:ff::a:1/128,1313:2222:3333:7cff::a:1/128,1111:2222:3333:7ffe::/127,1111:2222:3333:7ffc::/127,1212:2222:3333:7ffe::/127,1212:2222:3333:7ffc::/127,1414:2222:f024:1ff::/64,1111:2222:3333::/64,1111:2222:3333:7f02::/64,1414:2222:efca:7ffc::/64,1414:2222:efca:7f02::/64,1414:2222:efca:7eff::/64,1414:2222:efca:1::/64,1212:2222:3333:7f02::/64,1212:2222:3333:2::/64,1212:2222:3333:1::/64,1414:2222:ef98:7ffd::/64,1414:2222:ef98:7eff::/64,1414:2222:ef98:7bff::/64,1414:2222:ef98:7808::/64,1414:2222:ef98:7801::/64,1414:2222:ef98:110::/64,1414:2222:ef98:10c::/64,1515:2222:3333:8::/64,1515:2222:3333:7ffd::/64,1515:2222:3333:7eff::/64,1515:2222:3333:7dff::/64,1515:2222:3333:7d00::/64,1616:2222:3333:7ffd::/64,1616:2222:3333:7eff::/64,1616:2222:3333:7dff::/64,1616:2222:3333:7d00::/64,1616:2222:3333:7801::/64,1717:2222:3333:7eff::/64,1717:2222:3333:7dff::/64,1717:2222:3333:7d00::/64,1717:2222:3333:7806::/64,1717:2222:3333:7801::/64,1717:2222:3333:1fc::/64,1717:2222:3333:14::/64,1313:2222:3333:fe::/64,1313:2222:3333:8::/64,1313:2222:3333:7ffd::/64,1313:2222:3333:7eff::/64,1313:2222:3333:7dff::/64,1313:2222:3333:7d00::/64,1313:2222:3333:7cfe::/64,1313:2222:3333:7cfd::/64,1313:2222:3333:7800::/64,1313:2222:3333:11a::/64,1313:2222:3333:119::/64,1818:2222:3333:1007::/64,1919:2222:3333:a::/64,1111:2222:3333:7f00::/63,1414:2222:efca:7ffe::/63,1414:2222:efca:7f00::/63,1414:2222:efca:2::/63,1212:2222:3333:7f00::/63,1414:2222:ef98:7ffe::/63,1414:2222:ef98:7f00::/63,1414:2222:ef98:78fe::/63,1414:2222:ef98:7802::/63,1515:2222:3333:7ffe::/63,1515:2222:3333:7f00::/63,1616:2222:3333:7ffe::/63,1616:2222:3333:78fe::/63,1616:2222:3333:7802::/63,1717:2222:3333:7ffe::/63,1717:2222:3333:7f00::/63,1717:2222:3333:7804::/63,1717:2222:3333:7802::/63,1717:2222:3333:1fe::/63,1717:2222:3333:1f4::/63,1313:2222:3333:fc::/63,1313:2222:3333:a::/63,1313:2222:3333:7ffe::/63,1313:2222:3333:7f00::/63,1313:2222:3333:7c08::/63,1313:2222:3333:1fe::/63,1414:2222:ef98:fc::/62,1414:2222:ef98:7cfc::/62,1414:2222:ef98:7c00::/62,1414:2222:ef98:7b00::/62,1414:2222:ef98:7804::/62,1414:2222:ef98:1fc::/62,1414:2222:ef98:108::/62,1515:2222:3333:fc::/62,1616:2222:3333:fc::/62,1616:2222:3333:7f00::/62,1616:2222:3333:18::/62,1717:2222:3333:1f0::/62,1313:2222:3333:7804::/62,1313:2222:3333:110::/62,1414:2222:ef98::/61,1414:2222:ef98:100::/61,1515:2222:3333::/61,1616:2222:3333:10::/61,1717:2222:3333:100::/61,1313:2222:3333::/61,1313:2222:3333:7c00::/61,1313:2222:3333:7808::/61,1616:2222:3333::/60,1313:2222:3333:7810::/60,1313:2222:3333:100::/60,1313:2222:3333:7820::/59,1313:2222:3333:7840::/58,1313:2222:3333:7880::/57,1414:2222:ef98:7d00::/56,1313:2222:3333:7900::/56,1313:2222:3333:7a00::/55,1515:2222:3333:8000::/49,1616:2222:3333:8000::/49,1414:2222:efc4::/48,1414:2222:efc2::/48,1414:2222:ef84::/48,2020:2222:efba::/48,2020:2222:efb8::/48,2020:2222:efaf::/48,2020:2222:efa8::/48,2020:2222:efa7::/48,2020:2222:efa5::/48,2020:2222:ef96::/48,2121:2222:fbff::/48,2121:2222:eff5::/48,2121:2222:efeb::/48,2121:2222:efd8::/48,2121:2222:efad::/48,2121:2222:ef99::/48,2121:2222:ef94::/48,2121:2222:ef8f::/48,2121:2222:ef83::/48,2121:2222:ef7e::/48,2020:2222:efbc::/47,2121:2222:effc::/47,2121:2222:efe6::/47,10.136.52.248/32,10.82.219.222/32,10.248.96.52/32,10.248.96.51/32,10.80.125.220/32,10.102.146.94/32,10.102.135.134/32,10.245.252.3/32,10.167.120.90/32,10.103.158.138/32,10.196.90.224/32,10.181.40.227/32,10.214.148.202/32,10.213.167.130/32,10.211.171.56/32,10.152.17.50/32,10.39.67.131/32,1.78.202.38/32,10.104.157.86/32,10.154.62.120/32,100.227.14.57/32,100.193.202.110/32,100.193.202.107/32,100.229.129.47/32,100.215.241.6/32,100.3.73.34/32,100.124.189.106/32,100.107.8.22/32,100.166.222.235/32,100.54.227.161/32,100.171.234.22/32,100.219.217.52/32,100.167.191.129/32,100.81.194.190/32,100.142.237.81/32,100.101.113.49/32,100.243.43.86/32,100.243.43.82/32,100.108.98.72/32,100.108.98.66/32,100.108.98.65/32,100.108.98.116/32,10.136.52.244/31,10.82.223.206/31,10.82.219.220/31,10.248.96.54/31,10.248.96.48/31,10.102.143.184/31,10.102.141.2/31,10.238.168.144/31,10.211.65.184/31,10.211.65.180/31,10.211.171.58/31,10.211.171.48/31,1.71.142.114/31,100.144.236.24/31,100.115.101.242/31,100.4.164.28/31,100.171.234.20/31,100.243.43.88/31,100.243.43.84/31,100.243.43.80/31,100.108.98.120/31,100.108.98.118/31,10.82.219.216/30,10.248.96.40/30,10.102.143.188/30,10.30.90.128/30,10.77.183.108/30,10.196.65.52/30,10.211.65.176/30,10.211.171.52/30,100.115.97.216/30,100.115.96.84/30,100.115.96.200/30,100.115.101.48/30,100.84.209.128/30,100.84.209.124/30,100.84.209.12/30,100.4.164.24/30,100.171.234.24/30,100.171.234.16/30,100.219.217.48/30,100.108.98.68/30,100.108.98.112/30,100.241.99.112/30,100.34.180.76/30,10.82.219.208/29,10.102.143.176/29,10.238.168.152/29,10.59.192.112/29,10.48.115.136/29,10.17.120.200/29,10.107.207.144/29,10.120.145.144/29,10.122.217.120/29,10.103.142.192/29,10.196.91.240/29,10.134.2.168/29,10.127.51.184/29,10.127.32.192/29,10.154.62.112/29,100.144.236.16/29,100.186.1.128/29,100.118.100.224/29,100.84.210.8/29,100.4.164.16/29,10.136.51.0/28,10.136.37.128/28,10.200.61.240/28,10.112.218.0/28,10.102.145.112/28,10.102.143.160/28,10.238.168.128/28,10.211.65.160/28,10.211.171.32/28,10.154.8.224/28,10.154.62.96/28,10.154.57.0/28,100.144.236.0/28,100.115.103.16/28,100.229.128.144/28,100.4.164.0/28,100.171.234.0/28,100.219.217.32/28,100.243.43.64/28,100.108.98.96/28,100.216.221.64/28,100.241.99.96/28,10.138.3.64/27,10.136.52.64/27,10.136.37.160/27,10.248.96.0/27,10.213.163.128/27,10.195.4.96/27,10.196.90.192/27,10.154.54.192/27,10.154.54.160/27,10.123.32.224/27,100.144.236.32/27,100.214.8.224/27,100.178.9.96/27,10.136.52.128/26,10.136.52.0/26,10.248.96.64/26,10.30.243.0/26,10.195.63.128/26,10.123.32.128/26,100.192.1.0/26,100.173.250.64/26,100.108.98.192/26,10.136.37.0/25,10.248.96.128/25,10.238.169.128/25,10.196.91.0/25,100.192.1.128/25,10.137.166.0/24,10.136.53.0/24,10.136.36.0/24,10.248.110.0/24,10.238.183.0/24,10.238.170.0/24,10.46.168.0/24,10.30.242.0/24,10.195.65.0/24,10.228.35.0/24,100.99.254.0/24,100.14.212.0/24,100.46.19.0/24,100.174.4.0/24,100.0.0.0/24,100.125.135.0/24,100.243.250.0/24,100.108.105.0/24,100.104.191.0/24,10.137.138.0/23,10.136.54.0/23,10.136.38.0/23,10.136.224.0/23,10.248.108.0/23,1.3.34.0/23,10.30.240.0/23,10.72.118.0/23,10.6.34.0/23,10.196.70.0/23,100.82.118.0/23,100.67.66.0/23,100.83.248.0/23,100.83.216.0/23,100.104.188.0/23,10.137.124.0/22,10.30.196.0/22,10.147.116.0/22,10.180.144.0/22,10.142.236.0/22,10.195.88.0/22,10.209.164.0/22,100.110.236.0/22,100.89.120.0/22,100.86.4.0/22,100.93.196.0/22,100.10.188.0/22,10.139.248.0/21,10.138.232.0/21,10.136.200.0/21,10.136.0.0/21,10.147.120.0/21,10.228.176.0/21,100.0.16.0/21,10.147.96.0/20,10.228.160.0/20,100.145.48.0/20,100.131.48.0/20,100.126.224.0/20,10.137.32.0/19,10.6.192.0/19,100.168.0.0/16,100.18.0.0/15,100.16.0.0/12,100.128.0.0/10,100.64.0.0/10,172.0.0.0/8,fc00::/7]" 

I did a small optimization in https://github.com/OISF/suricata/pull/3762, but this doesn't yet address the main issue.

@Andy Wick are you able to test out current git master? A fix has been merged for ticket #6376, which I suspect is the same issue.