Suricata

Jason, any ideas on this?

Victor Julien wrote:

Jason, any ideas on this?

Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).

In your suricata.yaml you could have something like:

%YAML 1.1
---

foobar-vars: &foobar-vars !include foobar-vars.yaml

vars:
  address-groups:
    <<: *foobar-vars
    HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]" 

and a foobar-vars.yaml like:

%YAML 1.1
---

FOOBAR: "1.2.3.4/24" 
FOOBAR2: "5.6.7.8/24" 

resulting in the effective configuration of:

vars:
  address-groups:
    FOOBAR: "1.2.3.4/24" 
    FOOBAR2: "5.6.7.8/24" 
    HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]" 

We could probably do some other hacks in the config, but this is probably the most YAML-like solution.

Jason Ish wrote:

Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).

Would this also work with having 2 *.yaml files instead of the both vars in one, like in your example?

But good to know, that for the recent version 2.0.3 it's not available, so i will have to make a workaround :)

Implementing such a feature would be very helpful.

Andreas Herz wrote:

Jason Ish wrote:

Not with the current code, but we may want to revisit the YAML anchors and aliases work (https://github.com/jasonish/suricata/tree/dev-config-yaml-aliases).

Would this also work with having 2 *.yaml files instead of the both vars in one, like in your example?

Yes, it would look like this:

--- suricata.yaml ---

%YAML 1.1
---

foobar1: &foobar1 !include foobar1.yaml

foobar2: &foobar2 !include foobar2.yaml

vars:
  address-groups:
    <<: *foobar1
    <<: *foobar2
    HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]" 

--- foobar1.yaml ---

%YAML 1.1
---

FOOBAR1: "1.1.1.1/32" 

--- foobar2.yaml

%YAML 1.1
---

FOOBAR2: "2.2.2.2/24" 

Resulting in an effective suricata.yaml of:

vars:
  address-groups:
    FOOBAR1: "1.1.1.1/32" 
    FOOBAR2: "2.2.2.2/24" 
    HOME_NET: "[192.168.0.0/16,10.0.50.0/24,172.16.0.0/12,$FOOBAR,$FOOBAR2]" 

Also noticing that the required %YAML 1.1 gets kind of annoying after a while..

Jason Ish wrote:

Yes, it would look like this:

[...]

This would be perfect and exactly what we need :) Thanks

Andreas Herz wrote:

Jason Ish wrote:

Yes, it would look like this:

[...]

This would be perfect and exactly what we need :) Thanks

Just wanted to confirm that it's working as expected, couldn't find any bugs yet. Thanks a lot to ish :)

Thank you for the feedback Andreas !

Found a small bug. It looks like there needs to be a little more parsing the included config files. I can trigger a segfault.

In suricata.conf the last part with the rules definition:

#[...]
#after vars like HOME_NET etc.

monitorrules: &monitorrules !include include/monitorrules.yaml

rule-files:

    <<: *monitorrules

The problematic rules.yaml also has rule-files:

%YAML 1.1
---

rule-files:
 - http-events.rules
 - emerging-trojan.rules
 - emerging-scan.rules
 - emerging-user_agents.rules

The proper .yaml should just include the "-*.rules". So it's a faulty configuration but maybe a warning/error is better then a segfault :) no big deal if you configure correctly but still something to keep in mind.

Since i talked to ish who did also fix the issue already (see https://github.com/inliniac/suricata/pull/1297) i would encourage/vote for his general yaml patch to be merged in suricata. He explained to me that it might never be merged since it's a little to complex. But since it's in the yaml spec and it has no real negative aspects (as far as i can tell) i would prefer it going into suricata. Our use case shouldn't be that special, since generating config files and having more options to do so really helps a lot.
With this patch you can change small parts in the config by using several files and keeping most of it just static.

Isn't this fixed then by https://github.com/inliniac/suricata/pull/1352?

Andreas Herz wrote:

Isn't this fixed then by https://github.com/inliniac/suricata/pull/1352?

Does the segfaul still occur for you? If not, we can probably close this out. And talk about YAML anchors/aliases in a new issue.

Well i get no segfault but the include isn't working anymore either :p

End of default suricata.yaml changed to:

monitorrules: &monitorrules !include include/monitorrules.yaml

rule-files:

    <<: *monitorrules

File include/monitorrules.yaml:

%YAML 1.1
---
 - http-events.rules
 - emerging-trojan.rules
 - emerging-scan.rules
 - emerging-user_agents.rules

That worked with 2.0.6 but not with 3.0 anymore and results in:

21/2/2016 -- 00:23:15 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - Invalid rule-files configuration section: expected a list of filenames.

My bad it's just not in master, i need to merge the changes from Jason :p But if they work i still vote for it going into master.