Bug #9122
openfirewall: file.data auto-prior-accept bypass
Description
The filedata inspect engine (file_data keyword) returned NO_MATCH for every non-matching file, also when the transaction had already moved past the engine's progress — when no further file can arrive for that direction.
In firewall mode an LTE rule (accept:tx at some progress) becomes definitive only when the engine returns a definitive result. Because NO_MATCH never becomes definitive, the rule stays pending indefinitely, and the fail-closed default app policy for the hook is not applied. Traffic no rule accepts is not dropped.
The observable case is the streaming registration: the http2 request body. For an LTE rule allowing only request bodies matching %PDF: a request uploads a non-matching body and is closed (END_STREAM) while the response is still in progress — the stream tx is still open. The engine's eof condition (tx progress moved past request_data) holds, but the engine kept returning NO_MATCH. The rule never became definitive, so the fail-closed default (drop:flow) was not applied when the request became final; it applied only late (at tx/capture end) and without the default policy alert.
file_data is registered for the http1 request body, the http2 request stream body and the smtp request data. In http1 and smtp the engine eof coincides with the tx end state, so the tx-end path made the rule definitive pre-fix, masking the issue; the http2 request body is the observable case.
Fix: return CANT_MATCH_FILES once the tx progress moved beyond the engine's progress (no file, or all files scanned), making the rule definitive at request eof. The existing per-file state reset keeps the rule matchable if the tx grows another file.