Actions
Bug #2355
closedMissing events with PF_RING 7.1.0
Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
Affected Versions:
Effort:
Difficulty:
Label:
Description
After building Suricata (4.0.0 and later, didn't try 3.x) with PF_RING 7.1.0, EVE output does no longer contain app-level events, only flows, some alerts (only those that do not require app-layer decoding) and stats. The stats.log confirms that apparently app-layer decoding is limited:
AF_PACKET:
----------------------------------------------------------------------------------- Date: 12/11/2017 -- 13:52:26 (uptime: 0d, 00h 06m 04s) ------------------------------------------------------------------------------------ Counter | TM Name | Value ------------------------------------------------------------------------------------ capture.kernel_packets | Total | 686336033 capture.kernel_drops | Total | 234516595 decoder.pkts | Total | 452024541 decoder.bytes | Total | 326598481839 decoder.invalid | Total | 752032 decoder.ipv4 | Total | 450564379 decoder.ipv6 | Total | 13465 decoder.ethernet | Total | 465278010 decoder.tcp | Total | 413616530 decoder.udp | Total | 26575273 decoder.icmpv4 | Total | 8305400 decoder.icmpv6 | Total | 50 decoder.gre | Total | 275 decoder.vlan | Total | 422557560 decoder.teredo | Total | 12613 decoder.avg_pkt_size | Total | 722 decoder.max_pkt_size | Total | 1514 flow.tcp | Total | 2239259 flow.udp | Total | 1106157 flow.icmpv6 | Total | 3 defrag.ipv4.fragments | Total | 10385 defrag.ipv4.reassembled | Total | 4960 decoder.icmpv4.ipv4_unknown_ver | Total | 9 decoder.tcp.invalid_optlen | Total | 8 decoder.vlan.unknown_type | Total | 752015 tcp.sessions | Total | 1552824 tcp.pseudo | Total | 36994 tcp.syn | Total | 4537984 tcp.synack | Total | 3343138 tcp.rst | Total | 2687129 tcp.stream_depth_reached | Total | 3222 tcp.reassembly_gap | Total | 22404 tcp.overlap | Total | 148635 tcp.insert_list_fail | Total | 130 detect.alert | Total | 45 app_layer.flow.http | Total | 17014 app_layer.tx.http | Total | 33506 app_layer.flow.smtp | Total | 1195 app_layer.tx.smtp | Total | 1455 app_layer.flow.tls | Total | 111789 app_layer.flow.ssh | Total | 10 app_layer.flow.smb | Total | 551 app_layer.flow.dcerpc_tcp | Total | 554 app_layer.flow.dns_tcp | Total | 39 app_layer.tx.dns_tcp | Total | 57 app_layer.flow.enip_tcp | Total | 2 app_layer.tx.enip_tcp | Total | 4313 app_layer.flow.failed_tcp | Total | 25526 app_layer.flow.dns_udp | Total | 468444 app_layer.tx.dns_udp | Total | 1361992 app_layer.flow.enip_udp | Total | 189 app_layer.tx.enip_udp | Total | 194 app_layer.flow.failed_udp | Total | 637524 flow_mgr.closed_pruned | Total | 443247 flow_mgr.new_pruned | Total | 1945025 flow_mgr.est_pruned | Total | 13 flow.spare | Total | 8800 flow.tcp_reuse | Total | 429273 flow_mgr.flows_checked | Total | 409914 flow_mgr.flows_notimeout | Total | 402764 flow_mgr.flows_timeout | Total | 7150 flow_mgr.flows_timeout_inuse | Total | 247 flow_mgr.flows_removed | Total | 6903 flow_mgr.rows_checked | Total | 65536 flow_mgr.rows_skipped | Total | 39210 flow_mgr.rows_maxlen | Total | 39 tcp.memuse | Total | 164095088 tcp.reassembly_memuse | Total | 1385919392 dns.memuse | Total | 61260823 http.memuse | Total | 1740109925 flow.memuse | Total | 285200512
PF_RING:
------------------------------------------------------------------------------------ Date: 12/11/2017 -- 14:30:24 (uptime: 0d, 00h 05m 10s) ------------------------------------------------------------------------------------ Counter | TM Name | Value ------------------------------------------------------------------------------------ capture.kernel_packets | Total | 501027458 capture.kernel_drops | Total | 70551514 decoder.pkts | Total | 502718889 decoder.bytes | Total | 385563135748 decoder.invalid | Total | 1228617 decoder.ipv4 | Total | 501258422 decoder.ipv6 | Total | 11947 decoder.ethernet | Total | 517635913 decoder.tcp | Total | 464279594 decoder.udp | Total | 28373641 decoder.icmpv4 | Total | 7055425 decoder.icmpv6 | Total | 40 decoder.gre | Total | 202 decoder.vlan | Total | 502718889 decoder.vlan_qinq | Total | 327549685 decoder.teredo | Total | 11092 decoder.avg_pkt_size | Total | 766 decoder.max_pkt_size | Total | 1518 flow.tcp | Total | 914059 flow.udp | Total | 3199030 flow.icmpv6 | Total | 2 defrag.ipv4.fragments | Total | 7740 defrag.ipv4.reassembled | Total | 3703 decoder.icmpv4.ipv4_unknown_ver | Total | 6 decoder.tcp.invalid_optlen | Total | 6 decoder.vlan.unknown_type | Total | 1228605 tcp.sessions | Total | 535445 tcp.syn | Total | 3695043 tcp.synack | Total | 2811274 tcp.rst | Total | 2366381 detect.alert | Total | 34 app_layer.flow.dns_udp | Total | 1373463 app_layer.tx.dns_udp | Total | 1490375 app_layer.flow.enip_udp | Total | 170 app_layer.tx.enip_udp | Total | 197 app_layer.flow.failed_udp | Total | 1825397 flow_mgr.new_pruned | Total | 2855072 flow.spare | Total | 13371 flow_mgr.flows_checked | Total | 737245 flow_mgr.flows_notimeout | Total | 728821 flow_mgr.flows_timeout | Total | 8424 flow_mgr.flows_removed | Total | 8424 flow_mgr.rows_checked | Total | 65536 flow_mgr.rows_skipped | Total | 28340 flow_mgr.rows_maxlen | Total | 44 tcp.memuse | Total | 148958880 tcp.reassembly_memuse | Total | 3276800 dns.memuse | Total | 42790886 flow.memuse | Total | 371814208
It's interesting that the remanining flows always have no content towards the client and are tagged "state":"new","reason":"timeout". Here's an anonymized sample with all src IPs set to 1.1.1.1 and all dest IPs set to 2.2.2.2:
{"timestamp":"2017-12-11T16:55:15.342431+0000","flow_id":844118785026378,"event_type":"flow","src_ip":"1.1.1.1","src_port":34177,"dest_ip":"2.2.2.2","dest_port":161,"proto":"UDP","app_proto":"failed","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":95,"bytes_toclient":0,"start":"2017-12-11T16:54:44.223562+0000","end":"2017-12-11T16:54:44.223562+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
{"timestamp":"2017-12-11T16:55:15.342480+0000","flow_id":984877748197184,"event_type":"flow","src_ip":"1.1.1.1","src_port":161,"dest_ip":"2.2.2.2","dest_port":54333,"proto":"UDP","app_proto":"failed","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":101,"bytes_toclient":0,"start":"2017-12-11T16:54:44.137024+0000","end":"2017-12-11T16:54:44.137024+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
{"timestamp":"2017-12-11T16:55:15.342518+0000","flow_id":140560192272402,"event_type":"flow","src_ip":"1.1.1.1","src_port":161,"dest_ip":"2.2.2.2","dest_port":47842,"proto":"UDP","app_proto":"failed","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":93,"bytes_toclient":0,"start":"2017-12-11T16:54:44.161810+0000","end":"2017-12-11T16:54:44.161810+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
{"timestamp":"2017-12-11T16:55:15.342564+0000","flow_id":2110902209124558,"event_type":"flow","src_ip":"1.1.1.1","src_port":44909,"dest_ip":"2.2.2.2","dest_port":53,"proto":"UDP","app_proto":"dns","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":82,"bytes_toclient":0,"start":"2017-12-11T16:54:44.694478+0000","end":"2017-12-11T16:54:44.694478+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
{"timestamp":"2017-12-11T16:55:15.342610+0000","flow_id":703561685279778,"event_type":"flow","src_ip":"1.1.1.1","src_port":5273,"dest_ip":"2.2.2.2","dest_port":53,"proto":"UDP","app_proto":"dns","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":79,"bytes_toclient":0,"start":"2017-12-11T16:54:44.926754+0000","end":"2017-12-11T16:54:44.926754+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
{"timestamp":"2017-12-11T16:55:15.342644+0000","flow_id":1970211965416417,"event_type":"flow","src_ip":"1.1.1.1","src_port":53,"dest_ip":"2.2.2.2","dest_port":56298,"proto":"UDP","app_proto":"failed","flow":{"pkts_toserver":1,"pkts_toclient":0,"bytes_toserver":311,"bytes_toclient":0,"start":"2017-12-11T16:54:44.177121+0000","end":"2017-12-11T16:54:44.177121+0000","age":0,"state":"new","reason":"timeout","alerted":false}}
I have prepared a VM containing a build to reproduce the issue. SSH (port 3022, user:user) into the box available from
https://steinbiss.name/suri/Debian-PFRING.ova -- there's a setup there that should show the problem (see README.txt in /home/user).
Files
Actions