- Table of contents
- Rust Notes
Rust Notes¶
This page exists to keep notes (comments, observations, etc) with respect to using Rust in Suricata for certain functions such as protocol parsing.
2017-01-18 Populating a C struct from Rust (and integrating Rust into the build)¶
I wanted to try passing a pointer to a struct from C to Rust and having Rust populate it to avoid the use of opaque (from C) objects. The branch can be found here:
https://github.com/jasonish/suricata/tree/ish-rust-1it turned out to be relatively easy to do, and profiling showed that the overhead was not very high.
This branch also shows one way that a Rust library can be integrated into the build so extra git checkouts are not required, additionally the Rust code is compiled into a static library for use by Suricata.
2017-01-18 DNS Performance Test¶
This test used a simple DNS header parser for UDP DNS requests and responses. Profiling was turned on and a 140MB file of DNS traffic was used.
*** Rust App Layer IP ver Proto cnt min max avg -------------------- ------ ----- ---------- ------------ ------------ ----------- dns IPv4 17 926708 399 79881 1063 985.1m 100.00 Proto detect IPv4 17 926708 471 37353 1122 1.0b *** Master App Layer IP ver Proto cnt min max avg -------------------- ------ ----- ---------- ------------ ------------ ----------- dns IPv4 17 926709 384 212034 995 922.7m 100.00 Proto detect IPv4 17 926709 384 34758 765 709.2m
Debugging¶
We'll need to have a configure flag for suricata to force the non-release version of crates.
Gdb allows you to step through C and rust code nicely.
panic¶
In Rust you can use panic!("some explanation of why"); similar to the BUG_ON() macro in Suricata/C
Backtrace (partial):
#0 0x00000000005fcf9d in rust_panic ()
#1 0x00000000005e4973 in sys_common::unwind::begin_unwind_inner::hd4253a1812044857bct ()
#2 0x0000000000679a18 in nfs3_parser::sys_common::unwind::begin_unwind<&str> (msg=..., file_line=0x93d808 <nfs3::NfsTcpParser::parse_tcp_data_ts::_FILE_LINE::h5eeb7498016bdd39pMc>)
at src/libstd/sys/common/unwind/mod.rs:227
#3 0x0000000000684ea9 in nfs3_parser::nfs3::NfsTcpParser::parse_tcp_data_ts (self=0x7fffec5a9f40, i=...) at <std macros>:3
#4 0x0000000000686ff9 in nfs3_parser::nfs3::NfsTcpParser.RParser::parse (self=0x7fffec5a9f40, i=..., direction=0 '\000') at src/nfs3.rs:786
#5 0x00000000006871b1 in nfs3_parser::nfs3::r_nfstcp_parse (direction=0 '\000',
input=0x219e0c0 "<data omitted>"..., input_len=1448,
ptr=0x7fffec5a9f40) at src/rparser.rs:143
#6 0x000000000068712f in r_nfstcp_parse () at src/rparser.rs:131
#7 0x00000000004506dc in AppLayerParserParse (tv=<optimized out>, alp_tctx=<optimized out>, f=0x1bc0870, alproto=18, flags=<optimized out>, input=<optimized out>, input_len=<optimized out>)
at app-layer-parser.c:993
Getting info about a struct:
(gdb) f 3
#3 0x0000000000684ea9 in nfs3_parser::nfs3::NfsTcpParser::parse_tcp_data_ts (self=0x7fffec5a9f40, i=...) at <std macros>:3
3 in <std macros>
(gdb) print rpc_phdr
$1 = (struct RpcRequestPacketPartial *) 0x7ffff2a05448
(gdb) print *rpc_phdr
$2 = {hdr = {frag_is_last = true, frag_len = 32932, xid = 1041988801, msgtype = 0}, rpcver = 2, program = 100003, progver = 3, procedure = 7}
Profiling¶
Profiling with tools like perf should be done in --release mode, however this mode by default strips symbols.
To change, add:
[profile.release] debug = true
To the Cargo.toml.
Allow "cargo test" To Work¶
Once we start calling back into Suricata C code, "cargo test" may fail to compile as Suricata is not available as a library to link against. To keep your Rust tests workings, C functions can be declared as function pointers then registered at some point during Suricata startup, an example of this can be found in the DNS app-layer which uses a "context" object to pass the function pointers into the Rust app-layer. Some more general and global should be done though.
The context object can be seen here:
And its registered into Rust a few lines down at:
Safely calling a callback from C is done here:
Vendoring Cargo/Rust Dependencies¶
Dependencies specified in Cargo.toml are downloaded as needed. While this is likely fine for development, it is not ideal for the release tarballs, ideally those would contain all the dependent cargo source. "cargo-vendor" is a Cargo plugin that should allow us to pull down these dependencies during a make dist.
https://users.rust-lang.org/t/cargo-vendoring-now-on-nightly/6776
https://crates.io/crates/cargo-vendor
Useful links¶
https://locka99.gitbooks.io/a-guide-to-porting-c-to-rust/content/
https://jbendig.github.io/fix-rs/2017/01/24/how-to-optimize-rust-programs-on-linux/
Licensing¶
Using Crates creates risk of implicitly importing incompatible licenses. Example: nom (MIT) has an optional dependency regex (MIT/APLv2) which depends on thread-id (APLv2). Mixing our GPL with MIT is fine, but APLv2 is not compatible.
We probably need to lock specific versions of all crates we use and review the licensing of all dependences before upgrading.