Bug #623
closed1.4b2 core dump with pf_ring and PAE kernel
Description
When running under CentOS 5.8 kernel 2.6.18-308.16.1.el5PAE with PF_RING support compiled in, Suricata dumps core. If I boot into 2.6.18-308.16.1.el5, it runs without issue, but of course most of my RAM doesn't show up.
9/11/2012 -- 18:35:29 - <Info> - This is Suricata version 1.4beta2 RELEASE
9/11/2012 -- 18:35:29 - <Info> - CPUs/cores online: 24
9/11/2012 -- 18:35:29 - <Info> - Found an MTU of 1500 for 'eth5'
9/11/2012 -- 18:35:29 - <Info> - allocated 2097152 bytes of memory for the defrag hash... 65536 buckets of size 32
9/11/2012 -- 18:35:29 - <Info> - preallocated 65535 defrag trackers of size 108
9/11/2012 -- 18:35:29 - <Info> - defrag memory usage: 9174932 bytes, maximum: 33554432
9/11/2012 -- 18:35:29 - <Info> - AutoFP mode using default "Active Packets" flow load balancer
9/11/2012 -- 18:35:29 - <Info> - preallocated 2048 packets. Total memory 6316032
9/11/2012 -- 18:35:29 - <Info> - allocated 131072 bytes of memory for the host hash... 4096 buckets of size 32
9/11/2012 -- 18:35:29 - <Info> - preallocated 1000 hosts of size 76
9/11/2012 -- 18:35:29 - <Info> - host memory usage: 207072 bytes, maximum: 16777216
9/11/2012 -- 18:35:29 - <Info> - allocated 2097152 bytes of memory for the flow hash... 65536 buckets of size 32
9/11/2012 -- 18:35:29 - <Info> - preallocated 10000 flows of size 200
9/11/2012 -- 18:35:29 - <Info> - flow memory usage: 4097152 bytes, maximum: 33554432
9/11/2012 -- 18:35:29 - <Info> - using magic-file /usr/share/file/magic
9/11/2012 -- 18:35:29 - <Info> - Delayed detect disabled
9/11/2012 -- 18:35:31 - <Error> - [ERRCODE: SC_ERR_OPENING_RULE_FILE(41)] - opening rule file /opt/suricata/etc/rules/emerging-virus.rules: No such file or directory.
9/11/2012 -- 18:35:34 - <Info> - 44 rule files processed. 11959 rules successfully loaded, 0 rules failed
9/11/2012 -- 18:35:39 - <Info> - 11960 signatures processed. 727 are IP-only rules, 3663 are inspecting packet payload, 9182 inspect application layer, 0 are decoder event only
9/11/2012 -- 18:35:39 - <Info> - building signature grouping structure, stage 1: adding signatures to signature source addresses... complete
9/11/2012 -- 18:35:39 - <Info> - building signature grouping structure, stage 2: building source address list... complete
9/11/2012 -- 18:35:41 - <Info> - building signature grouping structure, stage 3: building destination address lists... complete
9/11/2012 -- 18:35:42 - <Warning> - [ERRCODE: SC_ERR_FOPEN(44)] - Error opening file: "/etc/suricata/threshold.config": No such file or directory
9/11/2012 -- 18:35:42 - <Info> - Core dump size set to unlimited.
9/11/2012 -- 18:35:42 - <Info> - fast output device (regular) initialized: fast.log
9/11/2012 -- 18:35:42 - <Info> - Using 2 live device(s).
9/11/2012 -- 18:35:42 - <Info> - Unable to find pfring config for interface eth4, using default value or 1.0 configuration system.
/root/suricata/suricata.sh: line 3: 16144 Segmentation fault (core dumped) LD_LIBRARY_PATH=/opt/suricata/lib:/opt/pfring/lib /opt/suricata/bin/suricata --pfring-int=eth4 --pfring-int=eth5 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /opt/suricata/etc/suricata.yaml
Core: http://www.incorrect.org/core.7007.bz2
Configured: ./configure --prefix=/opt/suricata --enable-pfring --with-libpfring-libraries=/opt/pfring/lib --with-libpfring-includes=/opt/pfring/include
It runs without issue if I compile it without PF_RING support.
Updated by Peter Manev over 11 years ago
Hi,
Without PAE kernel runs fine as well, correct?
What is your kernel version?
thanks
Updated by Peter Manev over 11 years ago
I am sorry - 2.6.18 - just saw that.
I noticed that you use two interfaces:
LD_LIBRARY_PATH=/opt/suricata/lib:/opt/pfring/lib /opt/suricata/bin/suricata --pfring-int=eth4 --pfring-int=eth5 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /opt/suricata/etc/suricata.yaml
Does the problem go away when you use just one?(with PAE)
Are the interfaces bridged?
thank you
Updated by Matt Carothers over 11 years ago
Yes, it still crashes if I use a single interface. It also still crashes if I don't use PF_RING. E.g. if I start it with -i instead of --pfrint-int.
Updated by Matt Carothers over 11 years ago
Oh, and the interfaces are not bridged. They are load balanced, however. Both interfaces will see traffic from the same clients, though an individual session will always stay on one interface or the other.
Updated by Eric Leblond over 11 years ago
Is it possible for you to provide the binary to download ?
Updated by Matt Carothers over 11 years ago
Sure, here you go: https://dl.dropbox.com/u/97846692/suricata-1.4b2-pfring.tar.bz2
That's Suricata itself as well as the pf_ring libraries and kernel module.
Updated by Eric Leblond over 11 years ago
OK. It looks like a PF_RING issue as the crash occurs inside pf_ring_open.
What is the version of PF_RING ?
There's something I don't like in the way you run suricata: you listen on 2 interface with the same cluster-id.
So here's two things I would like you to try: * Start suricata with one interface * Start suricata with the two interfaces and set the cluster-id to separate values inside the yaml.
For last run you should have something like this inside the YAML:
pfring:
- interface: eth4
threads: 1
cluster-id: 99
cluster-type: cluster_flow
- interface: eth5
threads: 1
cluster-id: 98
cluster-type: cluster_flow
By the way in this case, you should be able to start suricata with
suricata --pf-ring
Updated by Matt Carothers over 11 years ago
Eric Leblond wrote:
OK. It looks like a PF_RING issue as the crash occurs inside pf_ring_open.
What is the version of PF_RING ?
Ok, that's fixed it. I was using 5.5.0, which is the latest stable version. I compiled a new kernel module from svn, and that cleared up the problem. Thanks!
Updated by Eric Leblond over 11 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100