Bug #393
closedSuricata can only load maximum of 101 IP only rules
Description
Hi,
I'm not quite sure if this is a bug suricata configuration issues but I observed that when I tried to load a custom 398 IP based rules, Suricata always throws up errors and can only load 101 rules of the IP rules.
My 398 IP rules are like this:
drop ip [111.222.333.444/25,.........] any -> $HOME_NET any (msg:"DROP Traffic"; reference:url,www.mydomain.com; threshold: type limit, track by_dst, seconds 3600, count 1; classtype:misc-attack; flowbits:set,ET.Evil; flowbits:set,ET.DROPIP; sid:24040000; rev:2307;)
Please can someone be kind enough to point me in the right direction.
Thanks.
Files
Updated by Lambert Osas over 12 years ago
Just an update:
I intentionally reduced the IP rules to 101 and to my surprise ALL rules were loaded without any errors.
[11707] 1/1/2012 -- 13:06:29 - (util-ioctl.c:91) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'eth0'
[11707] 1/1/2012 -- 13:06:29 - (detect-pcre.c:128) <Info> (DetectPcreRegister) -- Using PCRE match-limit setting of: 3500
[11707] 1/1/2012 -- 13:06:29 - (detect-pcre.c:138) <Info> (DetectPcreRegister) -- Using PCRE match-limit-recursion setting of: 1500
[11707] 1/1/2012 -- 13:06:29 - (suricata.c:1429) <Info> (main) -- preallocated 50 packets. Total memory 157000
[11707] 1/1/2012 -- 13:06:29 - (flow.c:840) <Info> (FlowInitConfig) -- initializing flow engine...
[11707] 1/1/2012 -- 13:06:29 - (flow.c:932) <Info> (FlowInitConfig) -- allocated 524288 bytes of memory for the flow hash... 65536 buckets
of size 8
[11707] 1/1/2012 -- 13:06:29 - (flow.c:952) <Info> (FlowInitConfig) -- preallocated 10000 flows of size 176
[11707] 1/1/2012 -- 13:06:29 - (flow.c:954) <Info> (FlowInitConfig) -- flow memory usage: 2284288 bytes, maximum: 33554432
[11707] 1/1/2012 -- 13:06:29 - (detect.c:631) <Info> (SigLoadSignatures) -- 1 rule files processed. 101 rules succesfully loaded, 0 rules f
ailed
[11707] 1/1/2012 -- 13:06:29 - (detect.c:2420) <Info> (SigAddressPrepareStage1) -- 101 signatures processed. 101 are IP-only rules, 0 are i
nspecting packet payload, 0 inspect application layer, 0 are decoder event only
Updated by Peter Manev over 12 years ago
Hi,
assuming you have all the defaults in the yaml - there should not be a problem, I have loaded much more than that, but.. everyone has their specifics.
1. What Suri ver are you running? - stable , git?
2. Does it only load 101 rules and the others are err-ed, or it crashes and it does not load at anything above 101 rules?
3. what does suricata.log says (usually by default in /var/log/suricata.log ) - are there any rules that fail to load and why?
4. Rule "grammar" is correct, for each and every rule?
5. Have you seen a different behaviour with a different Suri version?
Thanks
Updated by Lambert Osas over 12 years ago
- File custom.rules custom.rules added
Hi,
Thanks for your reply.
1. I'm using suri 1.1.1 stable
2. Suri always loads 101 rules from my custom rule file. But strangely, it loads similar IP only rules from emerging threat with over 500 IP only rules without any errors. I have checked my custom rules making sure that the sid are unique but I cannot find any errors.
3. Suri just loads only 101 lines from my rule and throws up errors for the other. These are the errors normally displayed: - Error parsing signature
4. Suricata does not crash when I load these custom rules. It just loads only 101 only from the rule file and discards the rest. I don't know what is wrong. The custom rule I used looks very like the similar emerging threat IP only rules.
5. I have only tested suri 1.1.1 . I haven't used other versions yet.
Please find the custom IP only rule file I'm using and please kindly scan it and see if there is any problem with the rule grammar .
Updated by Peter Manev over 12 years ago
Thanks Lambert.
Personally wouldn't be able to look at it more thouroughly until this Sat - hope it is not of inconvenience.
But it "sounds" like somethig is wrong with the file/rule/grammar itself...
Updated by Lambert Osas over 12 years ago
Hi,
Thanks for your reply and efforts. There is nothing wrong with the rules as I was able to reduce the whole IP rules to 98 lines and they were all loaded without any errors. However, with lines more than 101, I get errors.
The rules were generated by a script and I'm 100% sure there are no errors.
Still don't know why suri rejects the rules if more than 101 lines
Updated by Lambert Osas over 12 years ago
Maybe the errors has to do with the machine RAM or CPU. Would be nice if someone can load the rules and see if they will give errors :))
Updated by Victor Julien over 12 years ago
- Priority changed from High to Normal
It loads the custom.rules file fine for me, both on 1.1.1 and 1.2dev: 472 signatures processed. 472 are IP-only rules.
Can you share a bit more info on your setup? CPU, memory, OS, etc?
Updated by Lambert Osas over 12 years ago
Thanks so much for testing. So it appears it is my machine specs that might have caused it.
Here is my machine specs:
OS: Centos 5.6
CPU: Dual Core ATOM
RAM: 4GB
Thanks for great support!
Updated by Victor Julien over 12 years ago
Interesting, I can reproduce it on a CentOS 5.5 vm. On 1.2dev it gives a sh*tload of duplicate sig errors though, so it's not quietly failing. Investigating...
Updated by Lambert Osas over 12 years ago
Yes. That was exactly the errors I was getting. Lots if sig Duplicate errors and Parsing errors.
Thanks
Updated by Lambert Osas over 12 years ago
By the way, I forgot to say. The machine I tested with was a dedicated server. Centos 5.6
Updated by Victor Julien over 12 years ago
- Status changed from New to Closed
- Assignee set to Victor Julien
- Target version set to 1.2rc1
- % Done changed from 0 to 100
Found the issue. There is a bug that causes really large sid numbers to be parsed improperly. This causes duplication errors based on the wrong sid.
I'm fixing this in the git master (to be 1.2 soon). A workaround is to use lower sid numbers. Losing one zero should be enough: 2404000467 -> 240400467. Or update to the git master.