Bug #1587: CPU usage very high and download speed slow - Suricata - Open Information Security Foundation

Actions

Copy link

Bug #1587

closed

CPU usage very high and download speed slow

Added by Samiux A over 8 years ago. Updated over 6 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Target version:

Affected Versions:

Effort:

Difficulty:

Label:

Description

When downloading about 4 files (size are about 300 to 500 MB each), the CPU usage goes very high and the download is very slow. I also find the following error :

[9617] 30/10/2015 -- 13:44:39 - (suricata.c:1073) <Notice> (SCPrintVersion) -- This is Suricata version 2.1dev (rev dcbbda5)
[9617] 30/10/2015 -- 13:44:39 - (app-layer-template.c:435) <Notice> (RegisterTemplateParsers) -- Template TCP protocol detection enabled.
[9617] 30/10/2015 -- 13:44:39 - (app-layer-template.c:454) <Notice> (RegisterTemplateParsers) -- No echo app-layer configuration, enabling echo detection TCP detection on port 7.
[9617] 30/10/2015 -- 13:44:39 - (app-layer-template.c:472) <Notice> (RegisterTemplateParsers) -- Registering Template protocol parser.
[9617] 30/10/2015 -- 13:44:40 - (detect-template-buffer.c:43) <Notice> (DetectTemplateBufferRegister) -- Template application layer detect registered.
[9617] 30/10/2015 -- 13:44:40 - (output-json-template.c:194) <Notice> (TmModuleJsonTemplateLogRegister) -- Template JSON logger registered.
[9618] 30/10/2015 -- 13:44:43 - (tm-threads.c:2001) <Notice> (TmThreadWaitOnThreadInit) -- all 8 packet processing threads, 4 management threads initialized, engine started.
[9618] 30/10/2015 -- 13:46:20 - (detect-engine.c:593) <Notice> (DetectEngineReloadThreads) -- rule reload starting
[9618] 30/10/2015 -- 13:46:28 - (detect-engine.c:747) <Notice> (DetectEngineReloadThreads) -- rule reload complete
[9618] 30/10/2015 -- 13:46:28 - (suricata.c:2388) <Notice> (main) -- Signature(s) loaded, Detect thread(s) activated.
[9622] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9621] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9622] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9621] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9622] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9621] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9623] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9623] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9622] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9623] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9621] 30/10/2015 -- 20:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9621] 30/10/2015 -- 20:01:11 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9623] 30/10/2015 -- 20:01:11 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9621] 30/10/2015 -- 20:01:12 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9622] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9622] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9620] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9620] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9622] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9621] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9623] 30/10/2015 -- 20:04:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9623] 30/10/2015 -- 20:18:11 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9620] 30/10/2015 -- 20:23:08 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9622] 30/10/2015 -- 20:23:08 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9620] 30/10/2015 -- 20:26:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9622] 30/10/2015 -- 20:26:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9622] 30/10/2015 -- 20:26:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9621] 30/10/2015 -- 20:26:34 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9622] 30/10/2015 -- 20:26:35 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9620] 30/10/2015 -- 20:26:35 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9623] 30/10/2015 -- 20:26:35 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[9621] 30/10/2015 -- 21:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[9622] 30/10/2015 -- 21:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9620] 30/10/2015 -- 21:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[9622] 30/10/2015 -- 21:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[9620] 30/10/2015 -- 21:01:10 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

Is offloding turned off? https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Common_Errors - the last paragraph.

Actions

Copy link

Updated by Samiux A over 8 years ago

ethtool -k p119p1

Features for p119p1:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]

Actions

Copy link

Updated by Alexander Gozman over 8 years ago

Hmm... Maybe it's related to kernel settings? Please check some parameters using the following commands:

sysctl net.core.rmem_default
sysctl net.core.rmem_max
sysctl net.core.wmem_default
sysctl net.core.wmem_max

The values displayed in an output may be not optimal. Try to change them to something like this:

sysctl -w net.core.netdev_max_backlog=250000
sysctl -w net.core.rmem_default=8388608
sysctl -w net.core.wmem_default=8388608
sysctl -w net.core.wmem_max=16777216
sysctl -w net.core.rmem_max=16777216
sysctl -w net.ipv4.tcp_rmem='65536 8388608 16777216'
sysctl -w net.ipv4.tcp_wmem='65536 8388608 16777216'

Start suricata again and check if it works properly.

Actions

Copy link

Updated by Peter Manev over 8 years ago

Samiux - do you have the same problem when trying to reproduce the issue with much smaller files?

Actions

Copy link

Updated by Samiux A over 8 years ago

@Alexander,

The values are more larger than you suggested at my system. I think it is not related.

@Peter Pan,

I think it is not related to download files. I upgraded to the latest github version 2.1dev (rev 86711a1) and I find that it happens on every hours.

[25170] 2/11/2015 -- 10:09:51 - (suricata.c:1073) <Notice> (SCPrintVersion) -- This is Suricata version 2.1dev (rev 86711a1)
[25171] 2/11/2015 -- 10:09:55 - (tm-threads.c:2001) <Notice> (TmThreadWaitOnThreadInit) -- all 8 packet processing threads, 4 management threads initialized, engine started.
[25171] 2/11/2015 -- 10:11:51 - (detect-engine.c:593) <Notice> (DetectEngineReloadThreads) -- rule reload starting
[25171] 2/11/2015 -- 10:11:57 - (detect-engine.c:747) <Notice> (DetectEngineReloadThreads) -- rule reload complete
[25171] 2/11/2015 -- 10:11:57 - (suricata.c:2388) <Notice> (main) -- Signature(s) loaded, Detect thread(s) activated.
[25176] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25175] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25178] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25176] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25178] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25178] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 11:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 11:16:36 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 12:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25178] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25178] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25176] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25176] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25177] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25177] 2/11/2015 -- 13:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25178] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 14:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25178] 2/11/2015 -- 14:07:36 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25178] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25177] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 15:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 15:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25176] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25176] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25175] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25175] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 16:00:54 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25178] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25177] 2/11/2015 -- 17:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25176] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25175] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25176] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25176] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25177] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25175] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25176] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25176] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 16: Message too long
[25177] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 17: Message too long
[25178] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25178] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long
[25175] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 15: Message too long
[25178] 2/11/2015 -- 18:00:53 - (source-af-packet.c:678) <Warning> (AFPWritePacket) -- [ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 18: Message too long

Actions

Copy link

Updated by Samiux A over 8 years ago

It also happens when browser is loading very busy sites, such as :

http://www.miui.com/download-290.html
http://infoman.teikav.edu.gr/~stpapad/WindowsInternalsPart16thEdition.pdf

Meanwhile, I have a rule to handle the ClamAV md5 checksum :

drop http any any -> any any (msg:"LOCAL Malicious file - Clamav MD5 Hash"; flow:established; fileext:!"iso"; filestore; filemd5:blacklist_md5; classtype: suspicious-filename-detect; sid:1000006; rev:1;)

Actions

Copy link

Updated by Samiux A over 8 years ago

Forgot to mention, there is no high CPU usage when I upgraded to version 2.1dev (rev 86711a1).

Actions

Copy link

Updated by Samiux A over 8 years ago

Sorry, I can only produce the error on http://www.miui.com/download-290.html but not the pdf link. The site (http://www.miui.com/download-290.html) is busy these hours as a new ROM is just released for download.

Actions

Copy link

Updated by Samiux A over 8 years ago

@Alexander,

When I use the kernel values that you suggested, all the error gone. Thanks a lot.

Actions

Copy link

#10

Updated by Samiux A over 8 years ago

I still see the error but not by visiting http://www.miui.com/download-290.html.

Actions

Copy link

#11

Updated by Alexander Gozman over 8 years ago

Samiux A wrote:

I still see the error but not by visiting http://www.miui.com/download-290.html.

I have no more ideas :( Maybe something's wrong with a default-packet-size option?

Probably it would be better if suricata could show the size of a message...

Actions

Copy link

#12

Updated by Samiux A over 8 years ago

The errors appear after the hard drive is heavy reading on every hour.

Actions

Copy link

#13

Updated by Samiux A over 8 years ago

I have 2 identical hardware and software as well as settings for the Suricata in IPS mode (inline). One is placed in front of router (pfsense, router with firewall only) and the other is placed behind the router. I am using Snorby too.

The traffic flow is heavy at the IPS that in front of the router compared with the IPS behind the router. However, the captioned "Message too long" error only appears on the IPS that in front of the router but not at the IPS behind the router.

The captioned error message only appears on hourly basis that is after a very heavy reading on the hard drive of the IPS. Does it caused by Snorby?

Any idea?

Actions

Copy link

#14

Updated by Alexander Gozman over 8 years ago

Samiux A wrote:

I have 2 identical hardware and software as well as settings for the Suricata in IPS mode (inline). One is placed in front of router (pfsense, router with firewall only) and the other is placed behind the router. I am using Snorby too.

The traffic flow is heavy at the IPS that in front of the router compared with the IPS behind the router. However, the captioned "Message too long" error only appears on the IPS that in front of the router but not at the IPS behind the router.

The captioned error message only appears on hourly basis that is after a very heavy reading on the hard drive of the IPS. Does it caused by Snorby?

Any idea?

Check the system CPU load and memory usage using 'top' or 'htop'. Also, if possible, try to turn snorby off for a while.

Actions

Copy link

#15

Updated by Samiux A over 8 years ago

The "message too long" problem has been fixed.

I set the "defrag: yes" at "af-packet" section of suricata.yaml to "defrag: no" to fix my problem.

Actions

Copy link

#16

Updated by Victor Julien about 8 years ago

Status changed from New to Assigned
Assignee set to Eric Leblond
Target version set to 70

So it appears that in AF_PACKET IPS mode with defrag enabled we can get packets bigger than the MTU that we can't forward.

Actions

Copy link

#17

Updated by Victor Julien over 6 years ago

Status changed from Assigned to Closed
Assignee deleted (~~Eric Leblond~~)
Target version deleted (70)

We track the af-packet IPS + defrag issue in #1778

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Suricata

Custom queries

Bug #1587

CPU usage very high and download speed slow

Updated by Alexander Gozman over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Peter Manev over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Alexander Gozman over 8 years ago

Updated by Samiux A over 8 years ago

Updated by Victor Julien about 8 years ago

Updated by Victor Julien over 6 years ago