Project

General

Profile

Actions

Bug #3509

closed

Behavior for tcp fastopen

Added by Jungho Yoon about 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:
Needs backport

Description

Hello team

Some rules did not work while checking some tests for tcp fastopen.
I want to get help because I may not have understood the behavior properly.

The testing method is not very different from the existing issues(Feature #1203). I just wanted to check for additional segmentation or reassembly

Test:

client ------ suricata(IPS mode) ----- server

1. The MTU of the client is 150.
2. I added http header to http_cli.py

def main(ip, port, tfo_mode, uri):
        endpoint = (ip, port)

        HTTP_REQ_SAFE = 'GET /index.php HTTP/1.1\r\nUser-Agent: Mozilla/5.0\r\nAccept: */*\r\n\r\n'
        HTTP_REQ_EVIL = 'GET /%s HTTP/1.1\r\nUser-Agent: czxt2s\r\nAccept: */*\r\nHost: 10.0.0.61\r\nConnection: Keep-Alive\r\n\r\n' % uri

3. Send stream from client to server
python2.7 tfo_http_cli.py --tfo 10.0.0.61 80 /a1a2a3a4a5a6a7a8a9a0b1b2b3b4b5b6b7b8b9b0c1c2c3c4c5c6c7c8c9c0d1d2d3d4d5d6d7d8d9d0

then, d2 of the SYN segment is finally divided and the d3 string appears in the next segment. (See pcap for details.)
Through the rules below I expected to be all detected, but it is not.

Test Rules:

alert tcp any any -> any any (msg:"tfo test1"; content:"d2"; sid:1;)
alert tcp any any -> any any (msg:"tfo test2"; content:"d3"; sid:2;)
alert tcp-pkt any any -> any any (msg:"tfo test3"; content:"d2"; sid:3;)
alert tcp-pkt any any -> any any (msg:"tfo test4"; content:"d3"; sid:4;)
alert tcp-stream any any -> any any (msg:"tfo test5"; content:"d2"; sid:5;)
alert tcp-stream any any -> any any (msg:"tfo test6"; content:"d3"; sid:6;)
alert tcp-stream any any -> any any (msg:"tfo test7"; content:"d2d3"; sid:7;)
alert tcp any any -> any any (msg:"tfo test8"; content:"d2d3"; sid:8;)
alert tcp any any -> any any (msg:"tfo test9"; http.uri; content:"d2d3"; sid:9;)
alert tcp any any -> any any (msg:"tfo test10";  content:"GET"; sid:10;)
alert tcp any any -> any any (msg:"tfo test11"; flags:S; content:"GET"; sid:11;)
alert tcp any any -> any any (msg:"tfo test12"; http.method; content:"GET"; sid:12;)
alert http any any -> any any (msg:"tfo test13"; http.method; content:"GET"; sid:13;)
alert tcp any any -> any any (msg:"tfo test14"; http.user_agent; content:"czx"; sid:14;)
alert tcp any any -> any any (msg:"tfo test15"; http.connection; content:"Keep-Alive"; sid:15;)
alert tcp any any -> any any (msg:"tfo test16"; http.host; content:"10.0.0.61"; sid:16;)
IPS mode Detecion:

02/29/2020-23:40:46.218274  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56182 -> 10.0.0.61:80
02/29/2020-23:40:46.219522  [**] [1:3:0] tfo test3 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.219522  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.221291  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.221291  [**] [1:4:0] tfo test4 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.221291  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.221292  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.221292  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80

Pcap Reading:
The results were not the same.
Some rules was not detected.I think the length of the packets is not enough to reassemble in reading mode.

[root@ms-suricata ~]# suricata -c /etc/suricata/suricata.yaml -k none --simulate-ips -r /root/fastopen_segmentation4.pcap ;cat fast.log
[1420] 1/3/2020 -- 00:01:31 - (suricata.c:1654) <Info> (ParseCommandLine) -- Setting IPS mode
[1420] 1/3/2020 -- 00:01:31 - (suricata.c:1083) <Notice> (LogVersion) -- This is Suricata version 5.0.2 RELEASE running in USER mode
[1420] 1/3/2020 -- 00:01:31 - (tm-threads.c:2169) <Notice> (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 4 management threads initialized, engine started.
[1420] 1/3/2020 -- 00:01:31 - (suricata.c:2916) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[1421] 1/3/2020 -- 00:01:31 - (source-pcap-file.c:372) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 13 packets, 1118 bytes
02/29/2020-23:40:46.219131  [**] [1:3:0] tfo test3 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.219131  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.217535  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56182 -> 10.0.0.61:80
02/29/2020-23:40:46.220763  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.220763  [**] [1:4:0] tfo test4 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80
02/29/2020-23:40:46.220763  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56184 -> 10.0.0.61:80

In normal case(non fastopen. See pcap for details.), tcp-pkt or tcp-stream was not a problem for detection. Rules were also detected except "tfo test11(sid:11)".

python2.7 tfo_http_cli.py 10.0.0.61 80 ............................/a1a2a3a4a5a6a7a8a9a0b1b2b3b4b5b6b7b8b9b0c1c2c3c4c5c6c7c8c9c0d1d2d3d4d5d6d7d8d9d0
(The normal case is also divided into segments based on d2 and d3.)

The recorded app-layer test was detected the same on my PC.
(https://github.com/OISF/suricata-verify/pull/110/commits/54d289d7ea7d1d90c616ebd70a4392a6e442072f)

However, if an HTTP transaction is split into a SYN segment and the next segment, some "http. *" Keywords does not match. (such as sid:9/14/15/16).

Test result:
1. In the fastopen case, SYN segments were detected when using the per-packet option, such as tcp-pkt or tcp-flags and not detected without the per-packet option.
2. I expected the SYN segment payload to reassembly with other segments payload in fastopen. but, raw-reassembly was not working.
3. In the fastopen case, some "http.*" keyword does not match. It was not detected by segmentation. "http.method" is not detected regardless of segmentation.

Are there any restrictions on fastopen unlike normal cases?
Please check the above. Thank you.


Files

ips_mode_suricata.yaml (69.6 KB) ips_mode_suricata.yaml Jungho Yoon, 02/29/2020 01:37 PM
non_fastopen_segmentation.pcap (889 Bytes) non_fastopen_segmentation.pcap Jungho Yoon, 02/29/2020 01:37 PM
pcapRead_suricata.yaml (69.4 KB) pcapRead_suricata.yaml Jungho Yoon, 02/29/2020 01:37 PM
fastopen_segmentation.pcap (1.32 KB) fastopen_segmentation.pcap Jungho Yoon, 02/29/2020 02:56 PM
tcp_fastopen_segmentation.pcap (2.5 KB) tcp_fastopen_segmentation.pcap Jungho Yoon, 03/01/2020 11:44 PM
tcp_normal_segmentation.pcap (1.73 KB) tcp_normal_segmentation.pcap Jungho Yoon, 03/01/2020 11:48 PM

Related issues 1 (0 open1 closed)

Copied to Suricata - Bug #3574: Behavior for tcp fastopenClosedJeff LucovskyActions
Actions #1

Updated by Jungho Yoon about 4 years ago

In the reading mode example above, fastopen_segmentation4.pcap and attached fastopen_segmentation.pcap are the same file.

Updated by Jungho Yoon about 4 years ago

Oh sorry. The upload file is invalid. Please check the new file. Reading mode was confirmed to be different from the wrong file. The rest is the same.

tcp_fastopen_segmentation.pcap IPS mode Detecion:

03/02/2020-08:37:37.394862  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56288 -> 10.0.0.61:80
03/02/2020-08:37:37.396472  [**] [1:3:0] tfo test3 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.396472  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397516  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397516  [**] [1:4:0] tfo test4 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397516  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397521  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397521  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80

tcp_fastopen_segmentation.pcap Pcap Reading:

[root@ms-suricata ~]# suricata -c /etc/suricata/suricata.yaml -k none --simulate-ips -r /root/tcp_fastopen_segmentation.pcap ;cat fast.log
[2955] 2/3/2020 -- 08:40:09 - (suricata.c:1654) <Info> (ParseCommandLine) -- Setting IPS mode
[2955] 2/3/2020 -- 08:40:09 - (suricata.c:1083) <Notice> (LogVersion) -- This is Suricata version 5.0.2 RELEASE running in USER mode
[2955] 2/3/2020 -- 08:40:10 - (tm-threads.c:2169) <Notice> (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 4 management threads initialized, engine started.
[2955] 2/3/2020 -- 08:40:10 - (suricata.c:2916) <Notice> (SuricataMainLoop) -- Signal Received.  Stopping engine.
[2956] 2/3/2020 -- 08:40:10 - (source-pcap-file.c:372) <Notice> (ReceivePcapFileThreadExitStats) -- Pcap-file module read 1 files, 20 packets, 2219 bytes
03/02/2020-08:37:37.394880  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56288 -> 10.0.0.61:80
03/02/2020-08:37:37.396501  [**] [1:3:0] tfo test3 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.396501  [**] [1:11:0] tfo test11 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397620  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397620  [**] [1:4:0] tfo test4 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397620  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397916  [**] [1:2:0] tfo test2 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
03/02/2020-08:37:37.397916  [**] [1:6:0] tfo test6 [**] [Classification: (null)] [Priority: 3] {TCP} 10.0.0.50:56290 -> 10.0.0.61:80
Actions #3

Updated by Victor Julien about 4 years ago

  • Status changed from New to Assigned
  • Assignee set to Victor Julien
Actions #4

Updated by Victor Julien about 4 years ago

  • Tracker changed from Support to Bug
  • Target version set to 6.0.0beta1
  • Label Needs backport added
Actions #6

Updated by Jeff Lucovsky almost 4 years ago

  • Copied to Bug #3574: Behavior for tcp fastopen added
Actions

Also available in: Atom PDF