Project

General

Profile

Actions

Bug #4183

open

Timestamps sometimes off by 2 hours on Windows

Added by Phil Rzewski almost 4 years ago. Updated 5 months ago.

Status:
New
Priority:
Normal
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

On a scratch Windows 2016 VM in Google Cloud, I installed https://nmap.org/npcap/dist/npcap-1.00.exe and then https://www.openinfosecfoundation.org/download/windows/Suricata-6.0.0-beta1-1-64bit.msi. I'm testing using the pcap at https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz after uncompressing it.

To test, I invoke Suricata like this:

"\Program Files\Suricata\suricata.exe" -r wrccdc.pcap

I've found that when my system timezone is set to default UTC or timezones that are UTC+xx:xx, the timestamp of the top event in eve.json is 2018-03-23T19:58:22.647908+0000. If I paste that into the ISO8601 box at https://www.timestamp-converter.com/, it translates to:

Date (UTC) Mar 23, 2018, 8:58:22 PM

But when the system timezone is set to timezones that are UTC-xx:xx (such as U.S. Pacific, Eastern, etc.), the timestamp of the top event in eve.json is 2018-03-23T12:58:22.647908-0900. Pasted into that tool, it translates to:

Date (UTC) Mar 23, 2018, 10:58:22 PM

For what it's worth, I first tried to repro the problem with
Suricata v5.0.3, but then I ran into the bug discussed at https://forum.suricata.io/t/eve-json-windows-timestamp-field-has-eastern-daylight-time-appended-to-timestamp/197 which led me to see to the change at https://github.com/OISF/suricata/pull/4976/commits/bbdc11842d324005567bac5eafdea5fd0fc62688, which is why I'd ended up testing with Suricata v6. Since that code happens to relate to timezones specifically on Windows I can't help but wonder if it's somehow related, but I'm not enough to a C developer to see on my own if there's a bug in there, so I'm pointing it out just in case it helps someone who sees this issue that better knows the code.

Here's the output of "suricata.exe --build-info" on that system:

---
26/11/2020 -- 09:43:23 - <Info> - Running as service: no
This is Suricata version 6.0.0-beta1 RELEASE
Features: PCAP_SET_BUFF HAVE_PACKET_FANOUT HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_NSS HAVE_LUA HAVE_LIBJANSSON TLS TLS_C11 RUST
SIMD support: none
Atomic intrinsics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 10.2.0, C version 201112
compiled with -fstack-protector
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: _Thread_local
compiled with LibHTP v0.5.33, linked against LibHTP v0.5.33

Suricata Configuration:
AF_PACKET support: no
eBPF support: no
XDP support: no
PF_RING support: no
NFQueue support: no
NFLOG support: no
IPFW support: no
Netmap support: no
DAG enabled: no
Napatech enabled: no
WinDivert enabled: yes

Unix socket enabled:                     no
Detection enabled: yes
Libmagic support:                        no
libnss support: yes
libnspr support: yes
libjansson support: yes
hiredis support: no
hiredis async with libevent: no
Prelude support: no
PCRE jit: yes
LUA support: yes
libluajit: no
GeoIP2 support: yes
Non-bundled htp: no
Old barnyard2 support:
Hyperscan support: no
Libnet support: no
liblz4 support: yes
Rust support:                            yes
Rust strict mode: no
Rust compiler path: /mingw64/bin/rustc
Rust compiler version: rustc 1.43.0
Cargo path: /mingw64/bin/cargo
Cargo version: cargo 1.43.0
Cargo vendor: yes
Python support:                          yes
Python path: /mingw64/bin/python3
Python distutils yes
Python yaml yes
Install suricatactl: yes
Install suricatasc: yes
Install suricata-update: yes
Profiling enabled:                       no
Profiling locks enabled: no
Plugin support (experimental):           no

Development settings:
Coccinelle / spatch: no
Unit tests enabled: no
Debug output enabled: no
Debug validation enabled: no

Generic build parameters:
Installation prefix: /mingw64
Configuration directory: C:\Program Files\Suricata\
Log directory: C:\Program Files\Suricata\log

--prefix                                 /mingw64
--sysconfdir /mingw64/etc
--localstatedir /mingw64/var
--datarootdir /mingw64/share
Host:                                    x86_64-w64-mingw32
Compiler: gcc (exec name) / g++ (real)
GCC Protect enabled: yes
GCC march native enabled: no
GCC Profile enabled: no
Position Independent Executable enabled: no
CFLAGS -g -O2 -DOS_WIN32 -std=c11 -DNTDDI_VERSION=NTDDI_VISTA -D_WIN32_WINNT=_WIN32_WINNT_VISTA -I${srcdir}/../rust/gen -I${srcdir}/../rust/dist
PCAP_CFLAGS
SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
Actions #1

Updated by Phil Rzewski almost 4 years ago

Correction: I was incorrect when I said I couldn't trigger the problem on timezones that were UTC+xx:xx. When I was reproducing the problem, the UTC+xx:xx timezone I selected happened to be Tripoli, which is UTC+02:00, but also doesn't happen to observe daylight savings time. When I tried with the Amsterdam timezone setting that's +01:00 UTC but does observe daylight savings, that got me a timestamp in eve.json of 2018-03-23T20:58:22.825466+0200 which translates to Mar 23, 2018, 7:58:22 PM in the tool. So, it's incorrect, but in this case off by 1 hour and in the opposite direction. It looks like the problem tracks to use of daylight savings, but not being +/- UTC.

Actions #2

Updated by Peter Manev almost 4 years ago

Is it possible to share that pcap or some of it ?
Can this be reproduced on your system with any pcap ? And what WinOS do you you out of curiosity?

Actions #3

Updated by Phil Rzewski almost 4 years ago

Hi Peter. A sample public pcap that reproduces the issue for me is referenced in the opening remarks (https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz). I've reproduced the issue on Windows 10 Home, Windows 2016 Server, and Windows 2019 Server.

Actions #4

Updated by Peter Manev almost 4 years ago

I apologize - I totally missed that.
Will feedback after i run it.

Actions #5

Updated by Peter Manev almost 4 years ago

That pcap location is not downloadable/reachable - at least on my end.

Actions #6

Updated by Jason Ish almost 4 years ago

I can't either. Access denied, looks like Cloudflare is treating a direct link to that file as an attach?

Actions #7

Updated by Phil Rzewski almost 4 years ago

I'm not sure what to tell you. I've downloaded it multiple times over the past two years from my Mac laptop, Linux VMs, etc. Here's output of me just now downloading it on an Amazon AWS Ubuntu VM, for instance:

$ wget https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz
--2020-12-04 17:26:56--  https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz
Resolving archive.wrccdc.org (archive.wrccdc.org)... 104.31.84.114, 104.31.85.114, 172.67.197.243, ...
Connecting to archive.wrccdc.org (archive.wrccdc.org)|104.31.84.114|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 203308080 (194M) [application/octet-stream]
Saving to: ‘wrccdc.2018-03-23.010014000000000.pcap.gz’

wrccdc.2018-03-23.0 100%[===================>] 193.89M  69.8MB/s    in 2.8s    

2020-12-04 17:26:59 (69.8 MB/s) - ‘wrccdc.2018-03-23.010014000000000.pcap.gz’ saved [203308080/203308080]
Actions #8

Updated by Phil Rzewski almost 4 years ago

A colleague of mine in Switzerland confirmed that he's seeing problems similar to what you describe. Very odd! Well, to get around it, I've staged a copy of the same file on a public Google Cloud Storage bucket. This is the same file as referenced at that URL above, but renamed and uncompressed for simplicity:

https://storage.googleapis.com/brimsec/wrccdc.pcap

Actions #9

Updated by Philippe Antoine 5 months ago

  • Target version set to TBD
Actions #10

Updated by Philippe Antoine 5 months ago

  • Assignee set to Community Ticket
Actions

Also available in: Atom PDF