Suricata

Compiled and installed Suricata from git.
Started Suricata , waited /about 30 sec/ so it starts logging info in the "stats1.log".
Then I started the thsark dump, then visited the following websites:
www.cisco.com , www.youtube.com , www.cnn.com , www.facebook.com .

Please find attached http.log and tsharkdump pcap files.
I used the script (tcomppcaphttplog) previously uploaded to check for differences.

MOST of the http_get requests that were unmatched from the http.log in the tshark pcap file are actually there (in the tshark pcap), just the tshark pcap is missing the last letter/symbol.
However there are some http_get requests that were not found in the http.log.

Thanks

It appears that all missing requests from the http.log can be attributed to the following:

1. many of the connections are not closed properly: the entire FIN/RST sequence is missing.

2. some of the connections have missing packets causing the http parser to give up.

For (1) the reason we're not printing the request uri's is that currently we only consider a transaction ready for printing when the full transaction is received. For the final transaction in a connection we rely on the proper connection shutdown (either through FIN or RST) to reach this state.

The solution for this would be to keep separate track of requests that are ready for logging, despite the response part of the same transaction not being ready yet. For this we could hook into the "request done" callback.

Peter, can you revisit this with the git master? It should be way more accurate.

Checked
Suricata current git
Suricata Beta 3 rev 18da4a8
Suricata 1.0.6

Judging from the results we match 100% all http requests.

Thanks Peter.