I think it would be beneficial for Suricata to have an optional JSON formatted log file for alerts.
Some background: Right now the popular frontends that work with Suricata are dependent on Unified2 and Barnyard2 to get data from Suricata into a database. With JSON, there is more log file overhead but then this data can be more easily managed/forwarded with tools like syslog-ng, and then imported to MySQL or a non-traditional database like MongoDB.
We already use libjansson for the unix socket protocol, so using this would require some refactoring. Do you think this yajl will bring big benefits over libjansson?
Depends on what you use it for. Yajl allows for stream based parsing directly into your own structures. No need to parse everything into a tree of nodes and then read through that. So, I think yajl is more efficiently using resources, but at the cost of some extra code complexity sue to having to write callbacks for each node type. We needed the streaming parser for potentially large json structures coming in chunks. Yajl is nice, but I think libjannson is as well. They just solve different needs. For instance you could build libjannson on top of yajl.
The only performance critical use we have is generating JSON records, not parsing them. The unix socket code that parses JSON records is async and we expect low volume. Otherwise we will be using it to generate output based on alerts, events. So on the output side it is critical that it's fast.
If libjannson forces you to build a full in-memory json tree, then traverse the tree again to write it out, then yajl may be more performant as yajl can stream this to the output.