Project

General

Profile

Actions

Bug #4421

closed

flow manager: using too much CPU during idle (6.0.x backport)

Added by Jeff Lucovsky over 1 year ago. Updated about 15 hours ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

Less extreme variant of #4096: the new usleep based loops in the flow manager and recycler consume too much CPU time. In busy systems this has been show to be more efficient than the old pthread_condtimed logic, but in lower end systems the usleep approach has too much overhead.

The reason for getting rid of the pthread_condtimed logic was that it would actually wake up millions of times per second on busy systems, leading to lots of overhead.

Perhaps we can make the usleep value configurable. On lower end systems there is no need to wake up frequently. Or optionally bring back the old behavior. Overall I'm not a great fan these kinds of options.


Files

5.0.7 pidstat in vm measuring suricata process (5 KB) 5.0.7 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:56 PM
5.0.7 pidstat on host measuring pid vm running suricata (5.37 KB) 5.0.7 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:56 PM
5.0.10 pidstat in vm measuring suricata process (5.18 KB) 5.0.10 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:56 PM
5.0.10 pidstat on host measuring pid vm running suricata (5.71 KB) 5.0.10 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:56 PM
6.0.0 pidstat in vm measuring suricata process (5 KB) 6.0.0 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:56 PM
6.0.0 pidstat on host measuring pid vm running suricata (5.03 KB) 6.0.0 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:56 PM
6.0.1 pidstat in vm measuring suricata process (4.99 KB) 6.0.1 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:56 PM
6.0.1 pidstat on host measuring pid vm running suricata (5.03 KB) 6.0.1 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:56 PM
6.0.2 pidstat in vm measuring suricata process (5 KB) 6.0.2 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:56 PM
6.0.2 pidstat on host measuring pid vm running suricata (6.08 KB) 6.0.2 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:56 PM
6.0.3 pidstat on host measuring pid vm running suricata (5.38 KB) 6.0.3 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:57 PM
6.0.3 pidstat in vm measuring suricata process (5 KB) 6.0.3 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:57 PM
6.0.4 pidstat in vm measuring suricata process (5 KB) 6.0.4 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:57 PM
6.0.4 pidstat on host measuring pid vm running suricata (5.38 KB) 6.0.4 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:57 PM
6.0.5 pidstat in vm measuring suricata process (5 KB) 6.0.5 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:57 PM
6.0.5 pidstat on host measuring pid vm running suricata (5.38 KB) 6.0.5 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:57 PM
6.0.6 pidstat in vm measuring suricata process (5 KB) 6.0.6 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:57 PM
6.0.6 pidstat on host measuring pid vm running suricata (5.03 KB) 6.0.6 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:57 PM
6.0.6 usleep 100 pidstat in vm measuring suricata process (5.01 KB) 6.0.6 usleep 100 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:57 PM
6.0.6 usleep 100 pidstat on host measuring pid vm running suricata (5.04 KB) 6.0.6 usleep 100 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:57 PM
6.0.6 usleep 50000 pidstat in vm measuring suricata process (5.02 KB) 6.0.6 usleep 50000 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:58 PM
6.0.6 usleep 50000 pidstat on host measuring pid vm running suricata (5.04 KB) 6.0.6 usleep 50000 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:58 PM
6.0.6 usleep 1000000 pidstat in vm measuring suricata process (5.01 KB) 6.0.6 usleep 1000000 pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:58 PM
6.0.6 usleep 1000000 pidstat on host measuring pid vm running suricata (5.04 KB) 6.0.6 usleep 1000000 pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:58 PM
6.0.7-dev (f40ad90ad 2022-09-19) pidstat in vm measuring suricata process (5.08 KB) 6.0.7-dev (f40ad90ad 2022-09-19) pidstat in vm measuring suricata process Michiel Janssens, 09/23/2022 04:58 PM
6.0.7-dev (f40ad90ad 2022-09-19) pidstat on host measuring pid vm running suricata (5.11 KB) 6.0.7-dev (f40ad90ad 2022-09-19) pidstat on host measuring pid vm running suricata Michiel Janssens, 09/23/2022 04:58 PM
Baseline pidstat in vm measuring all (7.58 KB) Baseline pidstat in vm measuring all Michiel Janssens, 09/23/2022 04:58 PM
Baseline pidstat on host measuring pid vm (5.02 KB) Baseline pidstat on host measuring pid vm Michiel Janssens, 09/23/2022 04:58 PM
Stress-ng pidstat in vm measuring stress-ng cyclic usleep process (1.08 KB) Stress-ng pidstat in vm measuring stress-ng cyclic usleep process Michiel Janssens, 09/23/2022 04:58 PM
Stress-ng pidstat in vm running stressng cyclic usleep measuring all (6.67 KB) Stress-ng pidstat in vm running stressng cyclic usleep measuring all Michiel Janssens, 09/23/2022 04:58 PM
Stress-ng pidstat on host measuring pid vm running stress-ng cyclic usleep (5.02 KB) Stress-ng pidstat on host measuring pid vm running stress-ng cyclic usleep Michiel Janssens, 09/23/2022 04:59 PM
Summary usleep tests.ods (17.9 KB) Summary usleep tests.ods Michiel Janssens, 09/23/2022 04:59 PM
Baseline top in freebsd vm measuring all (1.41 KB) Baseline top in freebsd vm measuring all Michiel Janssens, 10/02/2022 01:24 PM
6.0.6 pidstat on host measuring pid freebsd vm running suricata (5.03 KB) 6.0.6 pidstat on host measuring pid freebsd vm running suricata Michiel Janssens, 10/02/2022 01:24 PM
6.0.6 vmstat in freebsd vm measuring all (974 Bytes) 6.0.6 vmstat in freebsd vm measuring all Michiel Janssens, 10/02/2022 01:24 PM
6.0.6 top in freebsd vm measuring suricata pid (1.71 KB) 6.0.6 top in freebsd vm measuring suricata pid Michiel Janssens, 10/02/2022 01:24 PM
6.0.8 pidstat on host measuring pid freebsd vm running suricata (5.03 KB) 6.0.8 pidstat on host measuring pid freebsd vm running suricata Michiel Janssens, 10/02/2022 01:24 PM
Baseline pidstat on host measuring pid freebsd vm (5.03 KB) Baseline pidstat on host measuring pid freebsd vm Michiel Janssens, 10/02/2022 01:24 PM
Baseline vmstat in freebsd vm measuring all (978 Bytes) Baseline vmstat in freebsd vm measuring all Michiel Janssens, 10/02/2022 01:24 PM
6.0.8 vmstat in freebsd vm measuring all (978 Bytes) 6.0.8 vmstat in freebsd vm measuring all Michiel Janssens, 10/02/2022 01:24 PM
6.0.8 top in freebsd vm measuring suricata pid (1.75 KB) 6.0.8 top in freebsd vm measuring suricata pid Michiel Janssens, 10/02/2022 01:24 PM
6.0.8 pidstat on host measuring pid vm running suricata (5.03 KB) 6.0.8 pidstat on host measuring pid vm running suricata Michiel Janssens, 10/02/2022 01:25 PM
Summary usleep tests with freebsd and 6.0.8.ods (19.3 KB) Summary usleep tests with freebsd and 6.0.8.ods Michiel Janssens, 10/02/2022 01:25 PM
6.0.8 pidstat in vm measuring suricata process (5 KB) 6.0.8 pidstat in vm measuring suricata process Michiel Janssens, 10/02/2022 01:25 PM

Related issues 1 (0 open1 closed)

Copied from Bug #4379: flow manager: using too much CPU during idleClosedVictor JulienActions
Actions

Also available in: Atom PDF