Project

General

Profile

Actions

Feature #5816

open

Exception policy stats counters

Added by Jamie Lavigne about 1 year ago. Updated 13 days ago.

Status:
Resolved
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

Exception policies support applying a default action to packets in various exceptional cases, but don't have related stats counters. I am interested in counters for each of the exception policies counting the number of times they are applied. The visibility these counters provide is important for understanding the source of these drops since they can not be reported in things like alert logs.

I think a single counter per exception policy counting the number of times it is invoked would be enough for the visibility we need. Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.


Subtasks 2 (1 open1 closed)

Feature #5890: Exception policy stats counters (6.0.x backport)RejectedActions
Feature #6509: Exception policy stats counters (7.0.x backport)AssignedJuliana Fajardini ReichowActions

Related issues 6 (4 open2 closed)

Related to Suricata - Feature #6230: stats: add drop reason countersClosedVictor JulienActions
Related to Suricata - Task #6443: Suricon 2023 brainstormAssignedVictor JulienActions
Related to Suricata - Task #6929: eve/stats: hide zero-values for counters individuallyIn ProgressJuliana Fajardini ReichowActions
Related to Suricata - Feature #6215: Exception policy log outputAssignedJuliana Fajardini ReichowActions
Has duplicate Suricata - Feature #5828: exceptions: add statsRejectedJuliana Fajardini ReichowActions
Blocks Suricata - Feature #6366: pop3 protocol detectionIn ReviewPhilippe AntoineActions
Actions #1

Updated by Jamie Lavigne about 1 year ago

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

Actions #2

Updated by Juliana Fajardini Reichow about 1 year ago

Actions #3

Updated by Juliana Fajardini Reichow about 1 year ago

  • Assignee changed from OISF Dev to Juliana Fajardini Reichow
Actions #4

Updated by Juliana Fajardini Reichow about 1 year ago

  • Target version changed from TBD to 7.0.0-rc2
Actions #5

Updated by Juliana Fajardini Reichow about 1 year ago

  • Status changed from New to In Progress

Jamie Lavigne wrote in #note-1:

Jamie Lavigne wrote:

Importantly, the counters should be enabled for all values of the exception policy (including "ignore") so that it's possible to know the impact they will have before they are enabled.

I suppose alternately an exception policy value like "count" would work for this.

Do you mean like besides `pass-flow`, `pass-packet`, `bypass`, `reject`, `drop-flow`, `drop-packet`, and `ignore`, there could also be a `count` value?
I'm thinking that the counters could be enabled, maybe under the Exception-policy master-switch.

I was thinking that the `exception-policy` stats counter could look something like:

{
   "exception_policy":{
      "totals":{
         "drop-flow":4,
         "drop-pkt":4,
         "pass-flow":3,
         "pass-pkt":0,
         "bypass":0,
         "ignore":15
      },
      "stream":{
         "memcap":{
            "policy":"drop-flow",
            "counter":0
         },
         "midstream":{
            "policy":"pass-flow",
            "counter":3
         },
         "reassembly_memcap":{
            "policy":"ignore",
            "counter":8
         }
      },
      "app_layer_error":{
         "policy":"drop-pkt",
         "counter":4
      },
      "defrag_memcap":{
         "policy":"drop-flow",
         "counter":4
      },
      "flow_memcap":{
         "policy":"ignore",
         "counter":7
      }
   }
}

Would this be: useful? Or too verbose?

Actions #6

Updated by Jamie Lavigne about 1 year ago

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

Actions #7

Updated by Juliana Fajardini Reichow about 1 year ago

Jamie Lavigne wrote in #note-6:

It would work - I'm not sure we would need the policy values like "policy":"drop-flow" echoed in the stats output though, as we will know the configured values of those already. I assume that the _delta counters will be supported for these as well?

Thanks for the quick feedback :)

About delta counters: At the moment, I don't see a reason why we can't have them. Thanks for the reminder!

Actions #8

Updated by OISF Ticketbot about 1 year ago

  • Subtask #5890 added
Actions #9

Updated by OISF Ticketbot about 1 year ago

  • Label deleted (Needs backport to 6.0)
Actions #10

Updated by Juliana Fajardini Reichow about 1 year ago

Current PR under review/discussion: https://github.com/OISF/suricata/pull/8735

Actions #11

Updated by Juliana Fajardini Reichow 11 months ago

To add: if exception policy is `ignore`, don't output to log events.

Actions #12

Updated by Juliana Fajardini Reichow 11 months ago

Actions #13

Updated by Juliana Fajardini Reichow 11 months ago

Actions #14

Updated by Juliana Fajardini Reichow 11 months ago

  • Target version changed from 7.0.0-rc2 to 8.0.0-beta1
Actions #15

Updated by Victor Julien 9 months ago

  • Related to Feature #6230: stats: add drop reason counters added
Actions #16

Updated by Juliana Fajardini Reichow 6 months ago

  • Related to Task #6443: Suricon 2023 brainstorm added
Actions #17

Updated by Juliana Fajardini Reichow 6 months ago

Mentioned during brainstorming as part of a larger discussion on how to debug stuff.

Actions #18

Updated by Juliana Fajardini Reichow 5 months ago

  • Label Needs backport to 7.0 added
Actions #19

Updated by OISF Ticketbot 5 months ago

  • Subtask #6509 added
Actions #20

Updated by OISF Ticketbot 5 months ago

  • Label deleted (Needs backport to 7.0)
Actions #21

Updated by Philippe Antoine about 1 month ago

Actions #22

Updated by Juliana Fajardini Reichow 16 days ago

  • Related to Task #6929: eve/stats: hide zero-values for counters individually added
Actions #23

Updated by Juliana Fajardini Reichow 13 days ago

  • Status changed from In Progress to Resolved
Actions #24

Updated by Juliana Fajardini Reichow 7 days ago

Actions

Also available in: Atom PDF