Suricata can only load maximum of 101 IP only rules
I'm not quite sure if this is a bug suricata configuration issues but I observed that when I tried to load a custom 398 IP based rules, Suricata always throws up errors and can only load 101 rules of the IP rules.
My 398 IP rules are like this:
drop ip [111.222.333.444/25,.........] any -> $HOME_NET any (msg:"DROP Traffic"; reference:url,www.mydomain.com; threshold: type limit, track by_dst, seconds 3600, count 1; classtype:misc-attack; flowbits:set,ET.Evil; flowbits:set,ET.DROPIP; sid:24040000; rev:2307;)
Please can someone be kind enough to point me in the right direction.
#1 Updated by Lambert Osas over 6 years ago
Just an update:
I intentionally reduced the IP rules to 101 and to my surprise ALL rules were loaded without any errors.
 1/1/2012 -- 13:06:29 - (util-ioctl.c:91) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'eth0'
 1/1/2012 -- 13:06:29 - (detect-pcre.c:128) <Info> (DetectPcreRegister) -- Using PCRE match-limit setting of: 3500
 1/1/2012 -- 13:06:29 - (detect-pcre.c:138) <Info> (DetectPcreRegister) -- Using PCRE match-limit-recursion setting of: 1500
 1/1/2012 -- 13:06:29 - (suricata.c:1429) <Info> (main) -- preallocated 50 packets. Total memory 157000
 1/1/2012 -- 13:06:29 - (flow.c:840) <Info> (FlowInitConfig) -- initializing flow engine...
 1/1/2012 -- 13:06:29 - (flow.c:932) <Info> (FlowInitConfig) -- allocated 524288 bytes of memory for the flow hash... 65536 buckets
of size 8
 1/1/2012 -- 13:06:29 - (flow.c:952) <Info> (FlowInitConfig) -- preallocated 10000 flows of size 176
 1/1/2012 -- 13:06:29 - (flow.c:954) <Info> (FlowInitConfig) -- flow memory usage: 2284288 bytes, maximum: 33554432
 1/1/2012 -- 13:06:29 - (detect.c:631) <Info> (SigLoadSignatures) -- 1 rule files processed. 101 rules succesfully loaded, 0 rules f
 1/1/2012 -- 13:06:29 - (detect.c:2420) <Info> (SigAddressPrepareStage1) -- 101 signatures processed. 101 are IP-only rules, 0 are i
nspecting packet payload, 0 inspect application layer, 0 are decoder event only
#2 Updated by Peter Manev over 6 years ago
assuming you have all the defaults in the yaml - there should not be a problem, I have loaded much more than that, but.. everyone has their specifics.
1. What Suri ver are you running? - stable , git?
2. Does it only load 101 rules and the others are err-ed, or it crashes and it does not load at anything above 101 rules?
3. what does suricata.log says (usually by default in /var/log/suricata.log ) - are there any rules that fail to load and why?
4. Rule "grammar" is correct, for each and every rule?
5. Have you seen a different behaviour with a different Suri version?
#3 Updated by Lambert Osas over 6 years ago
Thanks for your reply.
1. I'm using suri 1.1.1 stable
2. Suri always loads 101 rules from my custom rule file. But strangely, it loads similar IP only rules from emerging threat with over 500 IP only rules without any errors. I have checked my custom rules making sure that the sid are unique but I cannot find any errors.
3. Suri just loads only 101 lines from my rule and throws up errors for the other. These are the errors normally displayed: - Error parsing signature
4. Suricata does not crash when I load these custom rules. It just loads only 101 only from the rule file and discards the rest. I don't know what is wrong. The custom rule I used looks very like the similar emerging threat IP only rules.
5. I have only tested suri 1.1.1 . I haven't used other versions yet.
Please find the custom IP only rule file I'm using and please kindly scan it and see if there is any problem with the rule grammar .
#5 Updated by Lambert Osas over 6 years ago
Thanks for your reply and efforts. There is nothing wrong with the rules as I was able to reduce the whole IP rules to 98 lines and they were all loaded without any errors. However, with lines more than 101, I get errors.
The rules were generated by a script and I'm 100% sure there are no errors.
Still don't know why suri rejects the rules if more than 101 lines
#12 Updated by Victor Julien over 6 years ago
- Status changed from New to Closed
- Assignee set to Victor Julien
- Target version set to 1.2rc1
- % Done changed from 0 to 100
Found the issue. There is a bug that causes really large sid numbers to be parsed improperly. This causes duplication errors based on the wrong sid.
I'm fixing this in the git master (to be 1.2 soon). A workaround is to use lower sid numbers. Losing one zero should be enough: 2404000467 -> 240400467. Or update to the git master.