Suricata

Not sure here if the solution is just not to log non-string messages, or define a limit to message length logged, or something else.

Juliana Fajardini Reichow wrote in #note-1:

Not sure here if the solution is just not to log non-string messages, or define a limit to message length logged, or something else.

The payload in the screenshot is a JPEG image transferred via MQTT, like a camera would, for example.

Note that there already is a parser option (app-layer.mqtt.max-msg-length) that restricts the maximum length of a message to be processed (otherwise the message is skipped; default is 1MB). Of course this is different from logging; one might want to have a large message processed for detection but not logged in its entirety. I'd be happy to add a maximum logging threshold here in the outputs configuration for mqtt.

I can imagine that a use case of the MQTT parser is to monitor transmission patterns rather than alert on content (I'm not aware of any related attacks, at least), so I at least would rather opt for completeness of content rather than log file frugality.

Sascha Steinbiss wrote in #note-2:

Juliana Fajardini Reichow wrote in #note-1:

Not sure here if the solution is just not to log non-string messages, or define a limit to message length logged, or something else.

The payload in the screenshot is a JPEG image transferred via MQTT, like a camera would, for example.

Note that there already is a parser option (app-layer.mqtt.max-msg-length) that restricts the maximum length of a message to be processed (otherwise the message is skipped; default is 1MB). Of course this is different from logging; one might want to have a large message processed for detection but not logged in its entirety. I'd be happy to add a maximum logging threshold here in the outputs configuration for mqtt.

Thanks for volunteering your time! :)

I can imagine that a use case of the MQTT parser is to monitor transmission patterns rather than alert on content (I'm not aware of any related attacks, at least), so I at least would rather opt for completeness of content rather than log file frugality.

I think that it makes sense that one may want to process the whole content, but optionally limit the log output. To your point of allowed for completeness, I wonder if it would make sense to log full contents in case an alert is triggered for it. - Or if this makes things unnecessarily overly complex.

OK, I'll come up with a PR.

https://github.com/OISF/suricata/pull/11053

For files/binary blobs, would it make sense to use the file tracking infrastructure in suri for those?

Victor Julien wrote in #note-6:

For files/binary blobs, would it make sense to use the file tracking infrastructure in suri for those?

I wouldn't think so. There is no direct notion of files or file transfer in the protocol, it's just that some payloads may happen to be data that could make sense as a standalone file. Question is, how would we detect these without the use of magic (expensive) or looking at the presence of MQTT5 properties that specify MIME types (depends on whether the sender includes that). In the latter case one would also again need to decide which of those types would be useful to extract to files.

I think this can also be closed since https://github.com/OISF/suricata/pull/11454 has been merged into master?