Project

General

Profile

Actions

Bug #1753

closed

cygwin: after pcap, engine "freezes" for a long time before exiting

Added by Marko Stojanovic about 8 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
Affected Versions:
Effort:
Difficulty:
Label:

Description

Platform : Tested on 2012, 64bit version.
Configuration : 8GB ram, 8 core Xeon CPU , run as a virtual machine on VMWare ESXi host
Suricata versions : Tested on Suricata 3.0.1RC1

Ran from a command line with Administrator privileges with next command parameters :

"suricata --runmode=single -v -c suricata.yaml -r ..\Users\Administrator\Downloads\maccdc2012_00013.pcap"

After the scan completes ( I noticed performance boost in comparison with 2.0.11 with single runmode option, tnx Victor and Peter), and afterscan info is displayed, at one point the process will freeze for some time, and then it will finnish. While it's "frozen", Memory and CPU time is still consumed as like it's doing something.

Full output is in the attachment,but the most important part is displayed (with comments inserted ) :

ENGINE STARTS
30/3/2016 -- 19:47:57 - <Notice> - all 1 packet processing threads, 4 management threads initialized, engine started.
30/3/2016 -- 19:47:57 - <Info> - No packets with invalid checksum, assuming checksum offloading is NOT used
30/3/2016 -- 19:49:34 - <Info> - pcap file end of file reached (pcap err code 0)
30/3/2016 -- 19:49:34 - <Notice> - Signal Received. Stopping engine.
ENGINE ENDS ( less than 2 minutes of scanning a 1GB pcap file )

AFTER SCAN STATS START SHOWING
30/3/2016 -- 19:52:28 - <Info> - 0 new flows, 0 established flows were timed out, 0 flows in closed state
30/3/2016 -- 19:52:28 - <Info> - preallocated 1024 packets. Total memory 2861056
30/3/2016 -- 19:52:28 - <Info> - time elapsed 271.217s
30/3/2016 -- 19:52:28 - <Info> - 1002314 flows processed
30/3/2016 -- 19:52:47 - <Notice> - Pcap-file module read 3190917 packets, 1022686575 bytes
30/3/2016 -- 19:52:47 - <Info> - Stream TCP processed 3056562 TCP packets
30/3/2016 -- 19:52:47 - <Info> - Fast log output wrote 1194 alerts
30/3/2016 -- 19:52:47 - <Info> - HTTP logger logged 2897 requests
30/3/2016 -- 19:53:05 - <Info> - ippair memory usage: 334144 bytes, maximum: 16777216
AND HERE SOMETHING HAPPENS, 12 MINUTES OF NOTHING
30/3/2016 -- 20:05:02 - <Info> - host memory usage: 326144 bytes, maximum: 16777216
30/3/2016 -- 20:05:02 - <Info> - cleaning up signature grouping structure... complete
PROCESS EXITS


Files

SuricataOUT.txt (6.56 KB) SuricataOUT.txt Marko Stojanovic, 03/30/2016 01:17 PM
Actions #1

Updated by Marko Stojanovic about 8 years ago

Here is some info of stack details on 2 active threads of process "suricata.exe" while it is in this freeze state.

Thread1: suricata.exe+0x1000

ntoskrnl.exe!KeWaitForSingleObject+0x8d6
ntoskrnl.exe!KeDelayExecutionThread+0x9bc
ntoskrnl.exe!KeWaitForSingleObject+0x1cf
ntoskrnl.exe!PoStartNextPowerIrp+0x809
ntoskrnl.exe!SeAccessCheck+0x280
ntoskrnl.exe!SeAccessCheck+0x4f1
cygwin1.dll!cfmakeraw+0x699
cygwin1.dll!cygwin_dll_init+0x114c
cygwin1.dll!setprogname+0x353b

Thread2 : cygwin1.dll!setprogname+0x27f0

ntoskrnl.exe!KeWaitForSingleObject+0x8d6
ntoskrnl.exe!RtlDeleteElementGenericTableAvl+0xc4e
ntoskrnl.exe!ObReferenceObjectByPointerWithTag+0xec2
ntoskrnl.exe!SeAccessCheck+0x1ef
ntoskrnl.exe!KeDelayExecutionThread+0xc32
ntoskrnl.exe!KeWaitForSingleObject+0x1cf
ntoskrnl.exe!ObWaitForMultipleObjects+0x9d3
ntoskrnl.exe!NtReadFile+0x663
ntoskrnl.exe!KeSaveStateForHibernate+0x2a33
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x536
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x222
wow64.dll!Wow64SystemServiceEx+0x26a
wow64.dll!Wow64LdrpInitialize+0x435
ntdll.dll!RtlWow64EnableFsRedirection+0x3b
ntdll.dll!LdrInitializeThunk+0xe
ntdll.dll!ZwReadFile+0xc
cygwin1.dll!sigfillset+0x1d9e
cygwin1.dll!setprogname+0x283c
cygwin1.dll!setprogname+0x34b2

Actions #2

Updated by Victor Julien about 8 years ago

  • Target version changed from 3.0.1RC1 to TBD
Actions #3

Updated by Andreas Herz over 7 years ago

  • Assignee set to Anonymous
Actions #4

Updated by Victor Julien over 7 years ago

  • Status changed from New to Assigned
  • Assignee changed from Anonymous to Victor Julien
  • Target version changed from TBD to 70

This seems related to high number of locks we use. Playing with some optimizations.

Actions #5

Updated by Victor Julien about 7 years ago

  • Subject changed from After PCAP scanning, statistics engine "freezes" for a long time before the process exits to cygwin: after pcap, engine "freezes" for a long time before exiting
Actions #6

Updated by Andreas Herz almost 5 years ago

Can someone with this setup try to see if it's still an issue or should we "ignore" it since we recommend other ways to run suricata on Windows?

Actions #7

Updated by Victor Julien over 4 years ago

  • Status changed from Assigned to Closed
  • Assignee deleted (Victor Julien)
  • Target version deleted (70)

I think this is fixed in 5.0.

Actions

Also available in: Atom PDF