Feature #6422


dpdk: expand on DPDK allocation hints

Added by Lukas Sismis 6 months ago. Updated 3 months ago.

Target version:


It would be possible to pre-analyze the environment for more optimal performance. That includes the placement of NICs, CPU cores, and memory channels within respective NUMA nodes. The idea would be to centralize (put as much as possible on a single node).
Below is a pseudocode that drafts the idea. It could be possibly extended by analysis of occupied memory channels.

This ticket can also open discussion about whether Suri should hint "that" much. The analysis process then can get into some corner cases where the hardware architecture assumptions might not reflect what the user really wants - e.g. initial assumption to run Suricata primarily on a single NUMA node can be good for cases where users mistakenly mix CPU cores of multiple NUMA nodes but might not be preferred by more experienced user who really want to use all CPU cores from multiple NUMA nodes. But we still inform the user if the NIC is on different NUMA node so the user might just ignore this "suggestion" as well.

function hintHugepageAllocation():
    // Fetch CPU and NIC info from suricata.yaml
    // fetch only hardware NICs (NICs with PCIe address) 
    //      because in DPDK you can have virtual devices - possibly DPDK Bond 
    //      interfaces would be a special case because these are "virtual" 
    //      devices but usually encompass hardware NICs together

    cpu_cores, nics = fetchInfoFromSuricataYAML()

    // Query NUMA nodes for CPU cores and NICs
    numa_nodes_cpus = queryLSCPU()
    numa_nodes_nics = queryNICInfoFromSys()

    // Check NIC and CPU core placement
    if all nics are on the same numa node and cpu_cores span multiple numa nodes:
        // possibly the user want to use maximum core availability
        recommend("Move all CPU cores to the NUMA node where NICs are located.")

    else if nics and cpu_cores span multiple numa nodes:
        recommend("suggest to free hugepages on other numa nodes where nics and cpu_cores are not located")

    // Capture initial snapshot of hugepage state
    // a multidimensional table of numa nodes, hugepage size and alloced and available (free) count 
    initial_snapshot = captureHugepageSnapshot()

    // Initialize Suricata

    // Capture post-initialization snapshot
    post_init_snapshot = captureHugepageSnapshot()

    // Diff the snapshots
    diff = compareSnapshots(initial_snapshot, post_init_snapshot)

    for node in diff:
        if node has significant free hugepages:
            recommend("Decrease the number of hugepages on NUMA node " + node)

        else if node's hugepages are depleted and another node without any allocated cpu_core/NIC is using hugepages:
            recommend("Increase the hugepage count on depleted NUMA node " + node + ". DPDK prefers this node.")
Actions #1

Updated by Lukas Sismis 3 months ago

  • Priority changed from Normal to Low

Also available in: Atom PDF