Feature #1251
closedUsing more then one include file within one config part
Description
I want to achieve the following setup:
vars.yaml (inluded by suricata.yaml via "include: vars.yaml")
%YAML 1.1 --- vars: address-groups: HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]" EXTERNAL_NET: "!$HOME_NET" HTTP_SERVERS: "$HOME_NET" # and so on... port-groups: HTTP_PORTS: "80,3128,8000,8080" # and so on...
This file should include other .yaml files:
foobar.yaml
%YAML 1.1 --- FOOBAR: "1.2.3.4/24"
foobar2.yaml
%YAML 1.1 --- FOOBAR2: "5.6.7.8/24"
I played around with adding vars: address-groups: into the foobar.yaml, tried "!include" and "include:" but i never got the result i wanted.
The reason behind this is, that i want to fill the foobar(2).yaml with the current valid dynamic IPs (via network scripts) and then trigger a rule reload in suricata so the HOME_NET gets updated.
So is there a way to include several yaml files that add values within this hierarchy?
Updated by Jason Ish over 9 years ago
Victor Julien wrote:
Jason, any ideas on this?
Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).
In your suricata.yaml you could have something like:
%YAML 1.1 --- foobar-vars: &foobar-vars !include foobar-vars.yaml vars: address-groups: <<: *foobar-vars HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]"
and a foobar-vars.yaml like:
%YAML 1.1 --- FOOBAR: "1.2.3.4/24" FOOBAR2: "5.6.7.8/24"
resulting in the effective configuration of:
vars: address-groups: FOOBAR: "1.2.3.4/24" FOOBAR2: "5.6.7.8/24" HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]"
We could probably do some other hacks in the config, but this is probably the most YAML-like solution.
Updated by Andreas Herz over 9 years ago
Jason Ish wrote:
Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).
Would this also work with having 2 *.yaml files instead of the both vars in one, like in your example?
But good to know, that for the recent version 2.0.3 it's not available, so i will have to make a workaround :)
Implementing such a feature would be very helpful.
Updated by Jason Ish over 9 years ago
Andreas Herz wrote:
Jason Ish wrote:
Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).
Would this also work with having 2 *.yaml files instead of the both vars in one, like in your example?
Yes, it would look like this:
--- suricata.yaml ---
%YAML 1.1 --- foobar1: &foobar1 !include foobar1.yaml foobar2: &foobar2 !include foobar2.yaml vars: address-groups: <<: *foobar1 <<: *foobar2 HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]"
--- foobar1.yaml ---
%YAML 1.1 --- FOOBAR1: "1.1.1.1/32"
--- foobar2.yaml
%YAML 1.1 --- FOOBAR2: "2.2.2.2/24"
Resulting in an effective suricata.yaml of:
vars: address-groups: FOOBAR1: "1.1.1.1/32" FOOBAR2: "2.2.2.2/24" HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]"
Also noticing that the required %YAML 1.1 gets kind of annoying after a while..
Updated by Andreas Herz over 9 years ago
Jason Ish wrote:
Yes, it would look like this:
[...]
This would be perfect and exactly what we need :) Thanks
Updated by Andreas Herz over 9 years ago
Andreas Herz wrote:
Jason Ish wrote:
Yes, it would look like this:
[...]
This would be perfect and exactly what we need :) Thanks
Just wanted to confirm that it's working as expected, couldn't find any bugs yet. Thanks a lot to ish :)
Updated by Andreas Herz over 9 years ago
Found a small bug. It looks like there needs to be a little more parsing the included config files. I can trigger a segfault.
In suricata.conf the last part with the rules definition:
#[...] #after vars like HOME_NET etc. monitorrules: &monitorrules !include include/monitorrules.yaml rule-files: <<: *monitorrules
The problematic rules.yaml also has rule-files:
%YAML 1.1 --- rule-files: - http-events.rules - emerging-trojan.rules - emerging-scan.rules - emerging-user_agents.rules
The proper .yaml should just include the "-*.rules". So it's a faulty configuration but maybe a warning/error is better then a segfault :) no big deal if you configure correctly but still something to keep in mind.
Updated by Andreas Herz over 9 years ago
Since i talked to ish who did also fix the issue already (see https://github.com/inliniac/suricata/pull/1297) i would encourage/vote for his general yaml patch to be merged in suricata. He explained to me that it might never be merged since it's a little to complex. But since it's in the yaml spec and it has no real negative aspects (as far as i can tell) i would prefer it going into suricata. Our use case shouldn't be that special, since generating config files and having more options to do so really helps a lot.
With this patch you can change small parts in the config by using several files and keeping most of it just static.
Updated by Andreas Herz over 8 years ago
- Assignee set to Andreas Herz
- Target version set to 70
Updated by Victor Julien about 8 years ago
- Tracker changed from Support to Feature
Updated by Andreas Herz about 8 years ago
Isn't this fixed then by https://github.com/inliniac/suricata/pull/1352?
Updated by Jason Ish about 8 years ago
Andreas Herz wrote:
Isn't this fixed then by https://github.com/inliniac/suricata/pull/1352?
Does the segfaul still occur for you? If not, we can probably close this out. And talk about YAML anchors/aliases in a new issue.
Updated by Andreas Herz about 8 years ago
Well i get no segfault but the include isn't working anymore either :p
End of default suricata.yaml changed to:
monitorrules: &monitorrules !include include/monitorrules.yaml rule-files: <<: *monitorrules
File include/monitorrules.yaml:
%YAML 1.1 --- - http-events.rules - emerging-trojan.rules - emerging-scan.rules - emerging-user_agents.rules
That worked with 2.0.6 but not with 3.0 anymore and results in:
21/2/2016 -- 00:23:15 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - Invalid rule-files configuration section: expected a list of filenames.
Updated by Andreas Herz about 8 years ago
My bad it's just not in master, i need to merge the changes from Jason :p But if they work i still vote for it going into master.
Updated by Andreas Herz over 7 years ago
- Status changed from New to Closed
Not necessary anymore for the usecase