Project

General

Profile

Actions

Optimization #3829

open

pcap source: Counters, counters and... counters

Added by Roland Fischer over 3 years ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

There seem to be a lot of similar counters being tracked in source-pcap. Mentioned in #2845.

We shall revisit all the counters in source-pcap.

Comment from @victorjulien:

Also, we should consider the places where we count packets. I think I count 3 here: in livedev, in pkts/bytes and then in the stats api through capture_kernel*. And then we have of course the intermediate last_stats64 that you're introducing (for good reason). Thats a lot of stats keeping. W/o looking into why we do all this, my gut feel is that we can reduce some of this.

I agree that some of them seem redundant.

Actions #1

Updated by Roland Fischer over 3 years ago

Current Counters

Are:

  • pkts/bytes is the thread-local count incremented with each packet
  • stats API through capture_kernel_* are the thread-local counts in tv->perf_private_ctx. Periodically updated once a second with counts from last_stats64
  • livedev atomic counters for pkts and drops among other things. Seems to be process-wide counts if I got that right. Mixed update rate.
  • last_stats64 provides the thread-local 64bit pcap stats. It tracks the 32bit pcap counters and detects roll-overs. Periodically updated once a second with stats from pcap API.

pkts/bytes

Always provides the current view for the thread.

Tracks:

  • ptv->pkts
    Updated for every packet. Used for thread exit stats.
    However, also used if CHECKSUM_VALIDATION_AUTO enabled in ChecksumAutoModeCheck() to optimize those checks. Cannot replace with stats counters as the overhead would be too much, imho.
  • ptv->bytes
    Updated for every packet. Only used for thread exit stats.

stats API through capture_kernel_*

Provides an snapshot view for the thread in general. Only current when stats are polled, after that it's outdated but that does not matter. Updated once a second during stats polling from last_stats64

Tracks:

  • ptv->capture_kernel_packets
  • ptv->capture_kernel_drops
  • ptv->capture_kernel_ifdrops

livedev

Since these counts are atomic, and from the usage in the "auto checksum feature", I infer these are counters across all threads.

Tracks:

  • ptv->livedev->pkts
    Always provides the current view across all threads. Atomically updated for every packet. Used if CHECKSUM_VALIDATION_AUTO enabled.
  • ptv->livedev->drop
    Provides a snapshot view across all threads. Atomically updated once per second as part of the stats polling with ptv->last_stats64.ps_drop

last_stats64

Provides a snapshot view for the thread in general. Only current when stats are polled, after that it's outdated but that does not matter. Updated once a second during stats polling from 32bit pcap API.

Handles the 32bit roll-over. More efficient than misusing the stats API's counts for the same thing through StatsGetLocalCounterValue() and StatsAddUI64().

Will eventually go away once libpcap has a 64bit API.

Tracks:
  • ptv->last_stats64.ps_recv (packets)
  • ptv->last_stats64.ps_drop (dropped by libpcap)
  • ptv->capture_kernel_ifdrops (dropped by interface)
Actions #2

Updated by Philippe Antoine 9 months ago

  • Assignee set to Community Ticket
  • Target version set to TBD
Actions

Also available in: Atom PDF