Support #5768
closedsuricata-update failure on ubuntu 22.04
Description
~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
~$ suricata -V
This is Suricata version 6.0.9 RELEASE
~$ suricata-update -V
suricata-update version 1.2.6
~$ python3 -V
Python 3.10.6
after update python, suricata-update can't work on ubuntu 22.04,here is the err msg:
~$ sudo suricata-update
22/12/2022 -- 18:31:45 - <Info> -- Using data-directory /var/lib/suricata.
22/12/2022 -- 18:31:45 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
22/12/2022 -- 18:31:45 - <Info> -- Using /etc/suricata/rules for Suricata provided rules.
22/12/2022 -- 18:31:45 - <Info> -- Found Suricata version 6.0.9 at /usr/bin/suricata.
22/12/2022 -- 18:31:45 - <Info> -- Loading /etc/suricata/suricata.yaml
22/12/2022 -- 18:31:45 - <Info> -- Disabling rules for protocol http2
22/12/2022 -- 18:31:45 - <Info> -- Disabling rules for protocol modbus
22/12/2022 -- 18:31:45 - <Info> -- Disabling rules for protocol dnp3
22/12/2022 -- 18:31:45 - <Info> -- Disabling rules for protocol enip
Traceback (most recent call last):
File "/usr/bin/suricata-update", line 36, in <module>
sys.exit(main.main())
File "/usr/lib/suricata/python/suricata/update/main.py", line 1369, in main
sys.exit(_main())
File "/usr/lib/suricata/python/suricata/update/main.py", line 1197, in _main
files = load_sources(suricata_version)
File "/usr/lib/suricata/python/suricata/update/main.py", line 941, in load_sources
source_config = index.get_source_by_name(name)
File "/usr/lib/suricata/python/suricata/update/sources.py", line 138, in get_source_by_name
if name in self.index["sources"]:
TypeError: 'NoneType' object is not subscriptable
~$ sudo suricata-update update-sources
22/12/2022 -- 18:31:24 - <Info> -- Using data-directory /var/lib/suricata.
22/12/2022 -- 18:31:24 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
22/12/2022 -- 18:31:24 - <Info> -- Using /etc/suricata/rules for Suricata provided rules.
22/12/2022 -- 18:31:24 - <Info> -- Found Suricata version 6.0.9 at /usr/bin/suricata.
22/12/2022 -- 18:31:24 - <Info> -- Downloading https://www.openinfosecfoundation.org/rules/index.yaml
22/12/2022 -- 18:31:24 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
22/12/2022 -- 18:31:24 - <Info> -- Adding all sources
22/12/2022 -- 18:31:24 - <Info> -- Saved /var/lib/suricata/update/cache/index.yaml
Updated by Ben Shen over 1 year ago
I install suricata from this ppa
ppa:oisf/suricata-stable
Updated by Jason Ish over 1 year ago
- Assignee changed from Shivani Bhardwaj to Jason Ish
Hello, Sorry you are running into this issue. I am not able to replicate it on a minimal Ubuntu 22.04 install and I'm thinking its something to do with your environment that we are not handling correctly.
Could you tell me what language your system is in? This can be obtained with:
echo $LANG
or
cat /etc/locale.conf
And what does the contents of /var/lib/suricata/update/cache/index.yaml look like? It should look the same as https://www.openinfosecfoundation.org/rules/index.yaml. An intermittent network error might not be out of the question either, corrupting the downloading index.
Updated by Ben Shen over 1 year ago
- File generated by update-locale
LANG="en_US.UTF-8"
LC_NUMERIC="zh_CN.UTF-8"
LC_TIME="zh_CN.UTF-8"
LC_MONETARY="zh_CN.UTF-8"
LC_PAPER="zh_CN.UTF-8"
LC_NAME="zh_CN.UTF-8"
LC_ADDRESS="zh_CN.UTF-8"
LC_TELEPHONE="zh_CN.UTF-8"
LC_MEASUREMENT="zh_CN.UTF-8"
LC_IDENTIFICATION="zh_CN.UTF-8"
LANGUAGE="en_US:en"
~$ echo $LANG
zh_CN.UTF-8
i have check the file /var/lib/suricata/update/cache/index.yaml, it's empty.
:/var/lib/suricata/update/cache#ll
总用量 3412
drwxr-x--- 2 root root 4096 10月 6 23:02 ./
drwxr-xr-x 4 root root 4096 10月 5 03:11 ../rw-r--r- 1 root root 0 11月 26 05:59 1e596eb1ec1ec9b6d6121727d3d926b5-trafficid.rulesrw-r--r- 1 root root 0 11月 26 05:59 7cdf02cbcd8ef4d6a01f0292d88ff2ff-emerging.rules.tar.gzrw-r--r- 1 root root 0 11月 26 05:59 8b16e8b0dcfc723365cd771d486968ae-ja3_fingerprints.rulesrw-r--r- 1 root root 0 11月 26 05:59 92a59cfd53a431780ba3036effa26f9b-sslblacklist.rulesrw-r--r- 1 root root 3482934 10月 5 18:56 947fcb6bd57604135c7bc029f8cd04af-emerging.rules.tar.gzrw-r--r- 1 root root 0 11月 26 05:59 ad9883bf0206c59a6d1f97d958e0806a-hunting.rulesrw-r--r- 1 root root 0 11月 26 05:59 dab7c29437dd6ac2b52c41e3b29f1497-etn_aggressive.rulesrw-r--r- 1 root root 0 11月 26 05:59 edf8b1fd5fd0b05b0de4f79e492f06ed-malsilo.rules.tar.gzrw-r--r- 1 root root 0 12月 22 18:31 index.yaml
to solve this problem, i purge the suricata package and install the newest version from ppa again, but the same failure msg appears.
Updated by Ben Shen over 1 year ago
In /var/lib/suricata/update/cache/ I delete index.yaml ,and then get index.yaml again from https://www.openinfosecfoundation.org/rules/index.yaml
after done this, i run suricata-update again, it give this msg:
~$ sudo suricata-update
23/12/2022 -- 20:05:54 - <Info> -- Using data-directory /var/lib/suricata.
23/12/2022 -- 20:05:54 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
23/12/2022 -- 20:05:54 - <Info> -- Using /etc/suricata/rules for Suricata provided rules.
23/12/2022 -- 20:05:54 - <Info> -- Found Suricata version 6.0.9 at /usr/bin/suricata.
23/12/2022 -- 20:05:54 - <Info> -- Loading /etc/suricata/suricata.yaml
23/12/2022 -- 20:05:54 - <Info> -- Disabling rules for protocol http2
23/12/2022 -- 20:05:54 - <Info> -- Disabling rules for protocol modbus
23/12/2022 -- 20:05:54 - <Info> -- Disabling rules for protocol dnp3
23/12/2022 -- 20:05:54 - <Info> -- Disabling rules for protocol enip
23/12/2022 -- 20:05:54 - <Warning> -- Source index is older than 2 weeks. Please update with suricata-update update-sources.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://rules.emergingthreats.net/open/suricata-6.0.9/emerging.rules.tar.gz.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://security.etnetera.cz/feeds/etn_aggressive.rules.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://sslbl.abuse.ch/blacklist/ja3_fingerprints.rules.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://raw.githubusercontent.com/travisbgreen/hunting-rules/master/hunting.rules.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://sslbl.abuse.ch/blacklist/sslblacklist.rules.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://openinfosecfoundation.org/rules/trafficid/trafficid.rules.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Fetching https://malsilo.gitlab.io/feeds/dumps/malsilo.rules.tar.gz.
23/12/2022 -- 20:05:54 - <Error> -- 'latin-1' codec can't encode character '\u201c' in position 69: ordinal not in range(256)
23/12/2022 -- 20:05:54 - <Info> -- Done.
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/app-layer-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/decoder-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/dhcp-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/dnp3-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/dns-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/files.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/http-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/ipsec-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/kerberos-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/modbus-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/nfs-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/ntp-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/smb-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/smtp-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/stream-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loading distribution rule file /etc/suricata/rules/tls-events.rules
23/12/2022 -- 20:05:54 - <Info> -- Loaded 367 rules.
23/12/2022 -- 20:05:54 - <Info> -- Disabled 14 rules.
23/12/2022 -- 20:05:54 - <Info> -- Enabled 0 rules.
23/12/2022 -- 20:05:54 - <Info> -- Modified 0 rules.
23/12/2022 -- 20:05:54 - <Info> -- Dropped 0 rules.
23/12/2022 -- 20:05:54 - <Info> -- Enabled 0 rules for flowbit dependencies.
23/12/2022 -- 20:05:54 - <Info> -- Backing up current rules.
23/12/2022 -- 20:05:54 - <Info> -- Writing rules to /var/lib/suricata/rules/suricata.rules: total: 367; enabled: 311; added: 0; removed 0; modified: 0
23/12/2022 -- 20:05:54 - <Info> -- Writing /var/lib/suricata/rules/classification.config
23/12/2022 -- 20:05:54 - <Info> -- No changes detected, exiting.
Updated by Ben Shen over 1 year ago
I change my locale to en_US.UTF-8,but the same error msg appear again. i don't know what' wrong.
BTW
i'am just a mechanical engineer，not a programmer, so i don't know how to programe via python. i use suricata as my ips from 2013 becouse it easy to deploy on ubuntu.
This problem is really puzzled me.
Updated by Jason Ish over 1 year ago
This might be a long shot, but can you try downloading one of these rule files with curl, on the same machine that is running suricata-update and try to extract it? For example..
curl -O https://rules.emergingthreats.net/open/suricata-6.0.9/emerging.rules.tar.gz tar xvf emerging.rules.tar.gz
Updated by Ben Shen over 1 year ago
I have tried it before, but the same error. now i have to download rulers manually for updating
Updated by Jason Ish 11 months ago
Ben Shen wrote in #note-7:
I have tried it before, but the same error. now i have to download rulers manually for updating
Are you still having issues here? Ubuntu 22.04 is probably the most popular platform and we haven't been able to reproduce, or heard of others having the same issue.
Thanks.
Updated by Ben Shen 11 months ago
Jason Ish wrote in #note-8:
Ben Shen wrote in #note-7:
I have tried it before, but the same error. now i have to download rulers manually for updating
Are you still having issues here? Ubuntu 22.04 is probably the most popular platform and we haven't been able to reproduce, or heard of others having the same issue.
Thanks.
The issue is solved after upgrade to suricata 6.0.12.
Thank you.