Project

General

Profile

Actions

Bug #3664

closed

Removing filename ends with ".rules" requirement for URL feeds of rules

Added by Eoin Miller almost 4 years ago. Updated 24 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Affected Versions:
Effort:
Difficulty:
Label:

Description

suricata-update will only process contents of filenames that end in ".rules". However when using things like ThreatQ's exports of IDS signatures, there is no filename in the output. Please see the url value of the below source YAML.

Enabled source config:

checksum: false 
source: rapid7/open
url: https://<HOSTNAME>/api/export/<REDACTED-HASH>/?token=<REDACTED-TOKEN>

After running suricata-update, it will retrieve the ThreatQ feed of rules successfully and place them in the cache dir, specifically "d8431e4d1a8e4054452d35cdea90127a-". Since the filename is null, there is nothing after the MD5 and hyphen:
root@suricata:~# ls -laFh /var/lib/suricata/update/cache/
total 2.6M
drwxr-x--- 3 root root 4.0K Apr 24 00:26 ./
drwxr-xr-x 4 root root 4.0K Nov 20 23:58 ../
-rw-r--r-- 1 root root 2.5M Apr 24 00:27 1168f1cf2d4676c8d507bbb6ea3b2078-emerging.rules.tar.gz
-rw-r--r-- 1 root root 7.5K Apr 24 01:11 d8431e4d1a8e4054452d35cdea90127a-
-rw-r--r-- 1 root root 5.5K Apr 23 18:20 index.yaml
drwxr-xr-x 3 root root 4.0K Nov 20 22:49 tmp/

In suricata/update/main.py, we found that commenting out the last two lines in this snippet will cause the byte string value in the dict named files will be processed and added to the output appropriately:
...
for filename in sorted(files):
        if not filename.endswith(".rules"):    
            continue
...

Alternatively, it also may be possible to specify the output filename to be used in the source YAML config.

Actions #1

Updated by Shivani Bhardwaj almost 4 years ago

  • Status changed from New to Assigned
Actions #2

Updated by Jason Ish over 2 years ago

  • Target version set to 1.3.0
  • Affected Versions 1.2.3 added
Actions #3

Updated by Shivani Bhardwaj 9 months ago

  • Target version changed from 1.3.0 to TBD
Actions #4

Updated by Jason Ish 29 days ago

  • Status changed from Assigned to In Review
  • Assignee changed from Shivani Bhardwaj to Jason Ish
  • Target version changed from TBD to 1.3.1
Actions #6

Updated by Jason Ish 24 days ago

  • Status changed from In Review to Closed
Actions

Also available in: Atom PDF