Suricata

GrowRegionToSize does not seem to check that size is bigger than current region->buf_size

From the caller point of view start->buf_size may be bigger than mem_size even if if (start->buf_size < next->buf_size) as we have @mem_size = ToNextMultipleOf(next_re - slide_offset, cfg->buf_size);@norg

But looks painful to produce a pcap that will get to this part of the code with the right set of conditions/state...

Side question: how to fuzz efficiently this part of code ?

Philippe Antoine wrote in #note-2:

Side question: how to fuzz efficiently this part of code ?

A dedicated fuzz target?

Victor Julien wrote in #note-3:

Philippe Antoine wrote in #note-2:

Side question: how to fuzz efficiently this part of code ?

A dedicated fuzz target?

Sure, but I wonder how we can have a fuzz target focusing on the TCP engine

This wouldn't need the tcp engine at all I think, just the streaming buffer API.

StreamingBufferConfig cfg = { 8+0xb, 0xff, 8+0x3f, NULL, NULL, NULL };
StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x2b, 0x74);
StreamingBufferAppend(sb, &cfg, &seg1, segd, 0xfe);
StreamingBufferSlideToOffset(sb, &cfg, 0x6e);

=================================================================
6386ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606000001cea at pc 0x0001057f7072 bp 0x7ff7bc956ac0 sp 0x7ff7bc956290
WRITE of size 4294967087 at 0x606000001cea thread T0
#0 0x1057f7071 in __asan_memset+0x781 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0xdb071)
#1 0x103a56596 in GrowRegionToSize util-streaming-buffer.c:735
#2 0x103a49c31 in StreamingBufferSlideToOffset util-streaming-buffer.c:1025
#3 0x1035a9282 in LLVMFuzzerTestOneInput fuzz_streambuf.c:44

What do you think about it Victor ?

StreamingBufferConfig cfg = { 8+0x2f, 0x9, 8+1, NULL, NULL, NULL };
StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 1, 0x77);
StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x72, 0x6b);
StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x72, 0x1);

==10459==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x611000001a1c at pc 0x00010b3f7eee bp 0x7ff7b6cb2ae0 sp 0x7ff7b6cb22a0
WRITE of size 114 at 0x611000001a1c thread T0
    #0 0x10b3f7eed in __asan_memmove+0xe4d (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0xdbeed)
    #1 0x1096f4ec0 in StreamingBufferInsertAt util-streaming-buffer.c:1535

With STREAMING_BUFFER_CONFIG_INITIALIZER values:

    StreamingBufferConfig cfg1 = { 2048, 8, 262144, NULL, NULL, NULL };
    StreamingBuffer *sb = StreamingBufferInit(&cfg1);
    StreamingBufferInsertAt(sb, &cfg1, &seg1, segd, 15832832, 825239);
    StreamingBufferAppend(sb, &cfg1, &seg1, segd, 9934743);
    StreamingBufferSlideToOffset(sb, &cfg1, 9934743);

getting grow 6723584 <= 15832832 bs end thus the overflow

Better reproducer

    StreamingBufferConfig cfg = { 2048, 8, 262144, NULL, NULL, NULL };
    StreamingBufferSegment seg1;
    StreamingBuffer *sb = StreamingBufferInit(&cfg);
    uint8_t segd[0x100000];
    memset(segd, 'A', 256);

    if (size < 15)
        return 0;

    int r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x2000, 0);
    if (r != 0)
        return 0;
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x100, 0x221200);
    if (r != 0)
        return 0;
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0xFFFF0, 0x1e0000);
    if (r != 0)
        return 0;
    StreamingBufferSlideToOffset(sb, &cfg, 0x221212);

Ok, I am getting to the root cause of this :
RegionsIntersect is missing a case : the one where the new insertion contains strictly the existing region (regularized with max gap)

    int r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x2000, 0);
    if (r != 0)
        return 0;
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x100, 0xc0000);
    if (r != 0)
        return 0;
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x80802, 0x7FFFF);
    if (r != 0)
        return 0;
    StreamingBufferSlideToOffset(sb, &cfg, 0xc0000);

First we create a main region, ok

Then, we create a next small region at 0xc0000 = 3 times STREAMING_BUFFER_REGION_GAP_DEFAULT (262144)

We have
[ main offset:0 size:2000 offset:2000 ][ gap:be000 ][ aux offset:c0000 size:800 offset:100 ](max 2, cnt 2, sb->regions 2)

StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x80802, 0x7FFFF);. makes things go wrong

It is not merged with [ aux offset:c0000 size:800 offset:100 ]. as it should

Because we ar looking for first region matching 7ffff/100801

And we have @7ffff < 80000==reg_o==r->stream_offset - cfg->region_gap==c0000-40000 < reg_re==r->stream_offset + r->buf_size + cfg->region_gap==c0000+800+40000==100800 < 100801@norg

As we can see, this requires calling StreamingBufferInsertAt with data length >= 2* STREAMING_BUFFER_REGION_GAP_DEFAULT + bufsize=2048 + 2, so 526 338 bytes

If we want to see if region [C:D] intersects with [A:B] (given c<d and a<b)

1. a b c d : no match
2. a c b d : match ok
3. a c d b : should match but does not in current code
4. c a b d : match ok
5. c a d b : match ok
6. c d a b : no match

But only caller of StreamingBufferInsertAt is using uint16_t as data length, so always smaller than 512 kb !
By the way, it should be reflected in function prototype...

So TCP reproducer without exploiting bug in ListRegions

int r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x8000, 0);
    if (r != 0)
        return 0;
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x100, 0xc0000);
    if (r != 0)
        return 0;
    for (size_t i =1; i<14; i++) {
        r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0x8000, 0+0x8000*i);
        if (r != 0)
            return 0;
    }
    for (size_t i =1; 0x7FF*i<0x50000; i++) {
        StreamingBufferSlideToOffset(sb, &cfg, 0x7FF*i);
    }
    r = StreamingBufferInsertAt(sb, &cfg, &seg1, segd, 0xFFFF, 0x70000);
    if (r != 0)
        return 0;
    StreamingBufferSlideToOffset(sb, &cfg, 0x60000);

Analysis : we aim to have
start->buf_size > mem_size==next_re - slide_offset

So, we want to maximize start->buf_size while keeping a next region which must keep more than 256kbytes apart

We grow buf_size by multiple appending StreamingBufferInsertAt
Then, we StreamingBufferSlideToOffset while keeping buf_size by sliding just one byte less than cfg->buf_size and enjoying the rounding
But we stop before slide offset + buf_size reach next offset

We then append maximum data without getting the regions merged
We do a big slide, but not so big compared to buf_size, so that we reach next

Wondering if we can turn the StreamingBufferr API reproducer into a pcap by
- have a 3 way handshake
- Replace StreamingBufferInsertAt(length, offset) by a TCP packet with length and sequence number as offset
- Replace StreamingBufferSlideToOffset(offset) with TCP ack packet in the other direction with ack number as offset