Bug #6173

Updated by Victor Julien 12 months ago

Resolution of #5320 done in is causing a set of problems. 

 First, it breaks backward compatibility of events and users will have to upgrade all data handling for something that was there since ages. I think this could have been avoid. 

 Second, switching from a set of JSON @key: value@ to a dictionary @{"key": $key, "value": $value}@ is going to cause severe issues in lot of data lake. For example, let's take Elasticsearch case. If default template is kept, we are going to have the following issue. On an event with 2 headers at least: 
 "key: "alpha", "value": "jane" 
 "key": "beta", "value": "robert" 
 if user want to see if "alpha": "robert" is present then he will naturally do "key:alpha AND value:robert" and this will match on our event because of the way Elasticsearch handle array by default. If Elasticsearch is set up to use nested elements, the match should be correct. But I'm still completely missing how we can do a list of all values for a given header. Even more if we add the third point below. 

 Third, the key corresponding to header were "normalized" as it was a fixed string. This is not the case anymore as the key are not converted to lower case before being added to 'key'. The lack of conversion to lower case is making sense as this output is trying to match the reality of the transaction as close as possible.