Project

General

Profile

Actions

Feature #5816

closed
JL JF

stats: exception policy counters

Feature #5816: stats: exception policy counters

Added by Jamie Lavigne about 3 years ago. Updated 8 months ago.

Status:
Closed
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Exception policies support applying a default action to packets in various exceptional cases, but don't have related stats counters. I am interested in counters for each of the exception policies counting the number of times they are applied. The visibility these counters provide is important for understanding the source of these drops since they can not be reported in things like alert logs.

I think a single counter per exception policy counting the number of times it is invoked would be enough for the visibility we need. Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.


Subtasks 1 (0 open1 closed)

Feature #5890: Exception policy stats counters (6.0.x backport)RejectedActions

Related issues 7 (2 open5 closed)

Related to Suricata - Feature #6230: stats: add drop reason countersClosedVictor JulienActions
Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Task #6929: eve/stats: make stats API aware of meaningful zero-valuesAssignedJuliana Fajardini ReichowActions
Related to Suricata - Feature #6215: flow/output: log triggered exception policyClosedJuliana Fajardini ReichowActions
Related to Suricata - Optimization #7185: stats: exceptions: use search-friendly log outputClosedJuliana Fajardini ReichowActions
Related to Suricata - Feature #6509: stats: exception policy counters (7.0.x backport)ClosedJuliana Fajardini ReichowActions
Has duplicate Suricata - Feature #5828: exceptions: add statsRejectedJuliana Fajardini ReichowActions

JL Updated by Jamie Lavigne about 3 years ago Actions #1

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

JF Updated by Juliana Fajardini Reichow about 3 years ago Actions #2

JF Updated by Juliana Fajardini Reichow about 3 years ago Actions #3

  • Assignee changed from OISF Dev to Juliana Fajardini Reichow

JF Updated by Juliana Fajardini Reichow about 3 years ago Actions #4

  • Target version changed from TBD to 7.0.0-rc2

JF Updated by Juliana Fajardini Reichow about 3 years ago Actions #5

  • Status changed from New to In Progress

Jamie Lavigne wrote in #note-1:

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

Do you mean like besides `pass-flow`, `pass-packet`, `bypass`, `reject`, `drop-flow`, `drop-packet`, and `ignore`, there could also be a `count` value?
I'm thinking that the counters could be enabled, maybe under the Exception-policy master-switch.

I was thinking that the `exception-policy` stats counter could look something like:

{
   "exception_policy":{
      "totals":{
         "drop-flow":4,
         "drop-pkt":4,
         "pass-flow":3,
         "pass-pkt":0,
         "bypass":0,
         "ignore":15
      },
      "stream":{
         "memcap":{
            "policy":"drop-flow",
            "counter":0
         },
         "midstream":{
            "policy":"pass-flow",
            "counter":3
         },
         "reassembly_memcap":{
            "policy":"ignore",
            "counter":8
         }
      },
      "app_layer_error":{
         "policy":"drop-pkt",
         "counter":4
      },
      "defrag_memcap":{
         "policy":"drop-flow",
         "counter":4
      },
      "flow_memcap":{
         "policy":"ignore",
         "counter":7
      }
   }
}

Would this be: useful? Or too verbose?

JL Updated by Jamie Lavigne about 3 years ago Actions #6

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

JF Updated by Juliana Fajardini Reichow about 3 years ago Actions #7

Jamie Lavigne wrote in #note-6:

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

Thanks for the quick feedback :)

About delta counters: At the moment, I don't see a reason why we can't have them. Thanks for the reminder!

OT Updated by OISF Ticketbot about 3 years ago Actions #8

  • Subtask #5890 added

OT Updated by OISF Ticketbot about 3 years ago Actions #9

  • Label deleted (Needs backport to 6.0)

JF Updated by Juliana Fajardini Reichow almost 3 years ago Actions #10

Current PR under review/discussion: https://github.com/OISF/suricata/pull/8735

JF Updated by Juliana Fajardini Reichow almost 3 years ago Actions #11

To add: if exception policy is `ignore`, don't output to log events.

JF Updated by Juliana Fajardini Reichow almost 3 years ago Actions #12

JF Updated by Juliana Fajardini Reichow almost 3 years ago Actions #13

JF Updated by Juliana Fajardini Reichow almost 3 years ago Actions #14

  • Target version changed from 7.0.0-rc2 to 8.0.0-beta1

VJ Updated by Victor Julien over 2 years ago Actions #15

  • Related to Feature #6230: stats: add drop reason counters added

JF Updated by Juliana Fajardini Reichow over 2 years ago Actions #16

  • Related to Task #6443: Suricon 2023 brainstorm added

JF Updated by Juliana Fajardini Reichow over 2 years ago Actions #17

Mentioned during brainstorming as part of a larger discussion on how to debug stuff.

JF Updated by Juliana Fajardini Reichow over 2 years ago Actions #18

  • Label Needs backport to 7.0 added

OT Updated by OISF Ticketbot over 2 years ago Actions #19

  • Subtask #6509 added

OT Updated by OISF Ticketbot over 2 years ago Actions #20

  • Label deleted (Needs backport to 7.0)

PA Updated by Philippe Antoine about 2 years ago Actions #21

JF Updated by Juliana Fajardini Reichow almost 2 years ago Actions #22

  • Related to Task #6929: eve/stats: make stats API aware of meaningful zero-values added

JF Updated by Juliana Fajardini Reichow almost 2 years ago Actions #23

  • Status changed from In Progress to Resolved

JF Updated by Juliana Fajardini Reichow almost 2 years ago Actions #24

  • Related to Feature #6215: flow/output: log triggered exception policy added

PA Updated by Philippe Antoine almost 2 years ago Actions #25

JF Updated by Juliana Fajardini Reichow over 1 year ago Actions #26

VJ Updated by Victor Julien 12 months ago Actions #27

  • Subject changed from Exception policy stats counters to stats: exception policy counters

VJ Updated by Victor Julien 8 months ago Actions #28

  • Subtask deleted (#6509)

VJ Updated by Victor Julien 8 months ago Actions #29

  • Related to Feature #6509: stats: exception policy counters (7.0.x backport) added

VJ Updated by Victor Julien 8 months ago Actions #30

  • Status changed from Resolved to Closed
Actions

Also available in: PDF Atom