Project

General

Profile

Actions

Feature #4854

open

Feature #4566: pgsql: add subprotocol-states

pgsql: Add COPY subprotocol-state

Added by Juliana Fajardini Reichow over 3 years ago. Updated 2 days ago.

Status:
In Progress
Priority:
Normal
Target version:
Effort:
Difficulty:
Label:

Description

When the postgresql backend sends or receives the following commands:
- "COPY FROM STDIN"
- "COPY TO STDOUT"
It enters the COPY subprotocol-state, which comprises a different set of messages.

Currently, if Suri has the pgsql app-layer enabled and sees one of those messages, it doesn't know what to do and stops parsing pgsql messages from that flow altogether, afterwards.

PostgreSQL documentation: https://www.postgresql.org/docs/14/protocol-flow.html#PROTOCOL-COPY


Files

CopyInSubprotocol.png (36.2 KB) CopyInSubprotocol.png Juliana Fajardini Reichow, 04/01/2025 08:40 PM
CopyOutSubprotocol.png (42.1 KB) CopyOutSubprotocol.png Juliana Fajardini Reichow, 04/01/2025 08:40 PM

Subtasks 3 (2 open1 closed)

Feature #7644: pgsql: add CopyOut subprotocol/modeClosedJuliana Fajardini ReichowActions
Feature #7645: pgsql: add CopyIn subprotocol/modeNewJuliana Fajardini ReichowActions
Feature #7646: pgsql: add CopyBoth supprotocol/ modeNewJuliana Fajardini ReichowActions
Actions #1

Updated by Juliana Fajardini Reichow over 3 years ago

  • Subject changed from PGSQL app-layer-parser: Add COPY subprotocol to PGSQL app-layer-parser: Add COPY subprotocol-state
  • Description updated (diff)

Updating this to better word that these are postgresql subprotocols, which are somewhat interpreted as sub-states, by Suri.

Actions #2

Updated by Victor Julien over 3 years ago

  • Subject changed from PGSQL app-layer-parser: Add COPY subprotocol-state to pgsql: Add COPY subprotocol-state
Actions #3

Updated by Victor Julien over 2 years ago

  • Target version changed from 7.0.0-beta1 to 7.0.0-rc1
Actions #4

Updated by Victor Julien over 2 years ago

  • Target version changed from 7.0.0-rc1 to 8.0.0-beta1
Actions #5

Updated by Victor Julien 3 months ago

  • Priority changed from Normal to High
Actions #6

Updated by Juliana Fajardini Reichow 8 days ago

  • Status changed from New to In Progress
Actions #7

Updated by Juliana Fajardini Reichow 8 days ago · Edited

The COPY operations can be split into three different modes (cf https://www.postgresql.org/docs/13/protocol-flow.html#PROTOCOL-COPY):
- Copy-In mode - high-speed bulk data transfer to the server
- Copy-Out mode - high-speed bulk data transfer from the server
- Copy-Both - high-speed bulk data transfer to and from the server

Copy-In and Copy-Out operations will lead the backend (server) to switch to dedicated subprotocols that will last until the operation is completed (either with a Success or a Failure).
Those operations can be sent from either Simple Query or Extended Query modes.

Due to PostgreSQL protocol specifications, the Copy-Out sub-proto seems a bit easier to implement, as the backend will send one CopyData message per copied row, whereas there is no such guarantee in the specification for CopyData messages coming from the frontend.

So I'll start with the Copy-Out sub-protocol/ mode.

Actions #8

Updated by Juliana Fajardini Reichow 8 days ago

Simple-ish sequence diagram for the Copy-In mode

Actions #9

Updated by Juliana Fajardini Reichow 8 days ago

Simple-ish sequence diagram for the Copy-Out mode

Actions #10

Updated by Juliana Fajardini Reichow 5 days ago

Those weren't fully correct. Should update, maybe.

PR for CopyOut mode: https://github.com/OISF/suricata/pull/12947

Actions #11

Updated by Juliana Fajardini Reichow 3 days ago · Edited

Actions #12

Updated by Juliana Fajardini Reichow 2 days ago

  • Related to Feature #7644: pgsql: add CopyOut subprotocol/mode added
Actions #13

Updated by Juliana Fajardini Reichow 2 days ago

  • Target version changed from 8.0.0-beta1 to 8.0.0-rc1
Actions #14

Updated by Juliana Fajardini Reichow 2 days ago

  • Related to Feature #7645: pgsql: add CopyIn subprotocol/mode added
Actions #15

Updated by Juliana Fajardini Reichow 2 days ago

  • Target version changed from 8.0.0-rc1 to 8.0.0
Actions #16

Updated by Juliana Fajardini Reichow 2 days ago

  • Subtask #7646 added
Actions #17

Updated by Juliana Fajardini Reichow 2 days ago

  • Subtask #7645 added
Actions #18

Updated by Juliana Fajardini Reichow 2 days ago

  • Subtask #7644 added
Actions

Also available in: Atom PDF