Suricata

I bumped into this problem as well. I've been able to reproduce the problem with my favorite public data source, https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz.

Here's the steps of me reproducing this on a scratch Ubuntu 20.04 Linux VM.

$ sudo add-apt-repository -y ppa:oisf/suricata-stable
$ sudo apt update
$ sudo apt-get -y --no-install-recommends install suricata
$ sudo suricata-update
$ sudo chmod go+rx /var/lib/suricata/rules
$ wget https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz
$ gunzip wrccdc.2018-03-23.010014000000000.pcap.gz 

$ suricata -r wrccdc.2018-03-23.010014000000000.pcap 
2/4/2021 -- 16:13:18 - <Notice> - This is Suricata version 6.0.2 RELEASE running in USER mode
2/4/2021 -- 16:13:46 - <Notice> - all 3 packet processing threads, 4 management threads initialized, engine started.
2/4/2021 -- 16:14:05 - <Notice> - Signal Received.  Stopping engine.
2/4/2021 -- 16:14:11 - <Notice> - Pcap-file module read 1 files, 1650919 packets, 473586644 bytes

It seems that jq de-duplicates these silently, but it was flagged by the zq tool from the Zed platform (https://github.com/brimdata/zed).

$ zq eve.json > /dev/null
eve.json: duplicate field subject

I was then able to isolate the first instance of it at line 121 of my eve.json output.

$ head -121 eve.json | tail -1
{"timestamp":"2018-03-23T19:58:22.825466+0000","flow_id":1250013126474967,"pcap_cnt":2065,"event_type":"alert","src_ip":"10.47.2.156","src_port":49285,"dest_ip":"52.85.83.187","dest_port":443,"proto":"TCP","metadata":{"flowbits":["tcp.retransmission.alerted"],"flowints":{"tcp.retransmission.count":10}},"alert":{"action":"allowed","gid":1,"signature_id":2210054,"rev":1,"signature":"SURICATA STREAM excessive retransmissions","category":"Generic Protocol Command Decode","severity":3},"tls":{"subject":"C=US, ST=California, L=Mountain View, O=Mozilla Corporation, OU=Cloud Services, CN=*.services.mozilla.com","issuerdn":"C=US, O=DigiCert Inc, CN=DigiCert SHA2 Secure Server CA","subject":"C=US, ST=California, L=Mountain View, O=Mozilla Corporation, OU=Cloud Services, CN=*.services.mozilla.com","issuerdn":"C=US, O=DigiCert Inc, CN=DigiCert SHA2 Secure Server CA","serial":"01:7E:45:A3:1A:A5:0B:C3:50:53:BC:50:F9:B6:9B:AD","fingerprint":"db:52:69:e4:eb:9a:b5:e0:ce:26:94:0b:df:c4:b0:de:87:c8:4d:f1","sni":"activity-stream-icons.services.mozilla.com","version":"TLS 1.2","notbefore":"2017-10-03T00:00:00","notafter":"2020-01-08T12:00:00","ja3":{"hash":"0ffee3ba8e615ad22535e7f771690a28","string":"771,49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-13,29-23-24-25,0"},"ja3s":{"hash":"76cc3e2d3028143b23ec18e27dbd7ca9","string":"771,49199,0-65281-11-35-5-16"}},"app_proto":"tls","flow":{"pkts_toserver":51,"pkts_toclient":68,"bytes_toserver":4688,"bytes_toclient":79238,"start":"2018-03-23T19:58:22.802007+0000"}}

Putting that through https://jsonlint.com/ and adding emphasis, we see the problem:

{
    "timestamp": "2018-03-23T19:58:22.825466+0000",
    "flow_id": 1250013126474967,
    "pcap_cnt": 2065,
    "event_type": "alert",
    "src_ip": "10.47.2.156",
    "src_port": 49285,
    "dest_ip": "52.85.83.187",
    "dest_port": 443,
    "proto": "TCP",
    "metadata": {
        "flowbits": ["tcp.retransmission.alerted"],
        "flowints": {
            "tcp.retransmission.count": 10
        }
    },
    "alert": {
        "action": "allowed",
        "gid": 1,
        "signature_id": 2210054,
        "rev": 1,
        "signature": "SURICATA STREAM excessive retransmissions",
        "category": "Generic Protocol Command Decode",
        "severity": 3
    },
    "tls": {
--->        "subject": "C=US, ST=California, L=Mountain View, O=Mozilla Corporation, OU=Cloud Services, CN=*.services.mozilla.com",
        "issuerdn": "C=US, O=DigiCert Inc, CN=DigiCert SHA2 Secure Server CA",
--->        "subject": "C=US, ST=California, L=Mountain View, O=Mozilla Corporation, OU=Cloud Services, CN=*.services.mozilla.com",
        "issuerdn": "C=US, O=DigiCert Inc, CN=DigiCert SHA2 Secure Server CA",
        "serial": "01:7E:45:A3:1A:A5:0B:C3:50:53:BC:50:F9:B6:9B:AD",
        "fingerprint": "db:52:69:e4:eb:9a:b5:e0:ce:26:94:0b:df:c4:b0:de:87:c8:4d:f1",
        "sni": "activity-stream-icons.services.mozilla.com",
        "version": "TLS 1.2",
        "notbefore": "2017-10-03T00:00:00",
        "notafter": "2020-01-08T12:00:00",
        "ja3": {
            "hash": "0ffee3ba8e615ad22535e7f771690a28",
            "string": "771,49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-13,29-23-24-25,0" 
        },
        "ja3s": {
            "hash": "76cc3e2d3028143b23ec18e27dbd7ca9",
            "string": "771,49199,0-65281-11-35-5-16" 
        }
    },
    "app_proto": "tls",
    "flow": {
        "pkts_toserver": 51,
        "pkts_toclient": 68,
        "bytes_toserver": 4688,
        "bytes_toclient": 79238,
        "start": "2018-03-23T19:58:22.802007+0000" 
    }
}

I've attached the full EVE JSON output file as well.

PR is there: https://github.com/OISF/suricata/pull/6798

Side note: all alerts on TLS where subject or issuerdn was present would generate a duplicate making the log entry non JSON valid and unusable in some systems.

https://github.com/OISF/suricata/pull/6798

Looks like this also applies to 5?