Prior to change of fixes to ticket #6846, the application data is reported in almost every alerts, now the application data is barely included in the alert events.
Upon reviewing the recent updates, I've identified two significant changes that compromise the consistency and accuracy of application data reporting: 1. Inconsistent reporting for multiple alerts that belong to a single packet. With PACKET_ALERT_FLAG_TX, a UDP DNS packet will no longer include the DNS section in the connection alert, while the DNS alert will. Although this change may seem logical, it constitutes a breaking change in a minor version update.
2. Missing reporting of application data in TLS Alerts The application data for TLS (and potentially other protocols) is no longer reported in the TLS alert. As one can see from the below code, the sslsession is the alstate set to the flow, but it would never translates to PACKET_ALERT_FLAG_TX.
uint64_t txid = PACKET_ALERT_NOTX;
if ((alert_flags & PACKET_ALERT_FLAG_STREAM_MATCH) ||
(s->alproto != ALPROTO_UNKNOWN && pflow->proto == IPPROTO_UDP)) {
// if there is a stream match (TCP), or
// a UDP specific app-layer signature,
// try to use the last tx
if (pflow->alstate) {
txid = AppLayerParserGetTxCnt(pflow, pflow->alstate) - 1;
alert_flags |= PACKET_ALERT_FLAG_TX;
}
}
reject ip $HOME_NET any -> any any (msg:"Blocked - Has Blocked"; flow: to_server; flowbits:isset,HAS.BLOCKED.ALERT; noalert; classtype:block; sid:1; rev:1; gid:1111;)
reject tls $HOME_NET any -> any any (msg:"Blocked - TLS"; flow:to_server; flowbits:isnotset,HAS.ALLOWED.ALERT.TLS; flowbits:isnotset,HAS.BLOCKED.ALERT; luajit:tls.lua; flowbits:set,HAS.BLOCKED.ALERT; classtype:block-tls; sid:5; rev:2; gid:1111;)
reject dns $HOME_NET any -> any any (msg:"Blocked - DNS"; flow:to_server; luajit:dns.lua; flowbits:set,HAS.BLOCKED.ALERT; flowbits:set,HAS.BLOCKED.ALERT.DNS; classtype:block-dns; sid:2; rev:3; gid:131415;)
alert ip $HOME_NET any -> any any (msg:"Allowed Connection Attempt"; flow:to_server,not_established; flowbits:isset,CONNECTION_ATTEMPT; flowbits:isnotset,HAS.BLOCKED.ALERT; flowbits:set,HAS.ALLOWED.ALERT; flowbits:unset,CONNECTION_ATTEMPT; classtype:allow; sid:4; rev:5;)
alert dns $HOME_NET any -> any any (msg:"Access Control Allowed - DNS"; flow:to_server; flowbits:isnotset,HAS.BLOCKED.ALERT; flowbits:isnotset,HAS.BLOCKED.ALERT.DNS; dns.query; bsize:>0; flowbits:set,HAS.ALLOWED.ALERT; classtype:allow; sid:6; rev:4;)
alert tls $HOME_NET any -> any any (msg:"Allowed - TLS"; flow:to_server; flowbits:isnotset,HAS.BLOCKED.ALERT; flowbits:isnotset,HAS.ALLOWED.ALERT.TLS; flowbits:set,HAS.ALLOWED.ALERT.TLS; classtype:allow; sid:7; rev:2;)
So for the ip/connection alert, the PACKET_ALERT_FLAG_TX is not set, and thus no dns app data. I mentioned this might be the new expected behaviour as the ip rule does not contain the dns keyword, but it is still a backward incompatible change that would break many eve log dependent services. I don't see why we couldn't log the application data in this case, as it is a single packet.
I have tested the patch, the patch crashes when pflow is NULL. Since the condition would never get the flag set when pflow->alstate is NULL, I moved the pflow and pflow->alstate check outside, this is more readable than before and avoids the crashes. This fix now works for both of my cases.
if(pflow&&pflow->alstate){if((alert_flags&PACKET_ALERT_FLAG_STREAM_MATCH)||(s->alproto!=ALPROTO_UNKNOWN&&pflow->proto==IPPROTO_UDP)||(AppLayerParserGetTxCnt(pflow,pflow->alstate)==1)){// if there is a stream match (TCP), or// a UDP specific app-layer signature,// or only one transaction// try to use the good tx for the packet directionuint8_tdir=(p->flowflags&FLOW_PKT_TOCLIENT)?STREAM_TOCLIENT:STREAM_TOSERVER;txid=AppLayerParserGetTransactionInspectId(pflow->alparser,dir);alert_flags|=PACKET_ALERT_FLAG_TX;}}