Optimization #822
closed
Explore the performance impact of libhtp htp_list_t growing in size
Added by Anoop Saldanha over 11 years ago.
Updated over 6 years ago.
Description
libhtp uses a list data structure to hold various items. Explore the impact the growth of the list has on performance.
From this mail nn the oisf mailing list - "[Oisf-users] Help with 99% CPU usage", the lists have grown large and the bts indicate that the threads are spending a lot of time parsing these lists.
Files
I have attached the last thread-dumps (it occurs less times since the detect engine optimizations). This behaviour is also noticeable when shutting down Suricata, it seems there are way to many HTTP transactions.
I forgot to add that when running af-packet in cluster_cpu mode, it has more loss than when running in cluster_flow. The network interface affinity settings are as follows.
CPU0 CPU1 CPU2 CPU3
78: 153977205 0 0 0 PCI-MSI-edge eth1-rxtx-0
79: 8 142687894 0 0 PCI-MSI-edge eth1-rxtx-1
80: 1 0 153285742 0 PCI-MSI-edge eth1-rxtx-2
81: 1 0 0 177757021 PCI-MSI-edge eth1-rxtx-3
82: 0 0 0 0 PCI-MSI-edge eth1-event-4
I have also setup the interface ring buffers to the maximum supported.
# ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 512
- Status changed from New to Assigned
- Target version changed from 2.0rc2 to 2.0beta2
- Target version changed from 2.0beta2 to 2.0rc1
- Tracker changed from Bug to Optimization
- Target version changed from 2.0rc1 to TBD
- Assignee changed from Anoop Saldanha to OISF Dev
- Status changed from Assigned to Closed
- Assignee deleted (
OISF Dev)
- Target version deleted (
TBD)
Also available in: Atom
PDF