Fuzzing

Fuzzing is the process of providing random data to programs to identify unexpected behavior, such as crashes and panics.

This page covers fuzz testing, primarily with cargo-fuzz. Property testing with proptest is a related but distinct approach: it also generates random input, but runs as ordinary tests under cargo test and asserts that some property holds, rather than only searching for crashes and panics. See How to Write Property Tests below.

The following steps can be used in any Stellar contract workspace. If experimenting, try them in the increment example. The contract has an increment function that increases a counter value by one on every invocation.

How to Write Fuzz Tests

1. Install the nightly Rust toolchain. Nightly Rust is required to run cargo-fuzz.

   rustup install nightly

2. Install cargo-fuzz.

   cargo install --locked cargo-fuzz

3. Initialize a fuzz project by running the following command inside your contract directory.

   cargo fuzz init

4. Open the contract's Cargo.toml file. Add lib as a crate-type.

    [lib]
   -crate-type = ["cdylib"]
   +crate-type = ["lib", "cdylib"]

5. Open the generated fuzz/Cargo.toml file. Add the soroban-sdk dependency.

    [dependencies]
    libfuzzer-sys = "0.4"
   +soroban-sdk = { version = "*", features = ["testutils"] }

6. Open the generated fuzz/src/fuzz_target_1.rs file. It will look like the below.

   #![no_main]
   use libfuzzer_sys::fuzz_target;
   
   fuzz_target!(|data: &[u8]| {
       // fuzzed code goes here
   });

7. Fill out the fuzz_target! call with test setup and assertions. For example, for the increment example:

   #![no_main]
   use libfuzzer_sys::fuzz_target;
   use soroban_increment_with_fuzz_contract::{IncrementContract, IncrementContractClient};
   use soroban_sdk::{
       testutils::arbitrary::{arbitrary, Arbitrary},
       Env,
   };
   
   #[derive(Debug, Arbitrary)]
   pub struct Input {
       pub by: u64,
   }
   
   fuzz_target!(|input: Input| {
       let env = Env::default();
       let id = env.register(IncrementContract, ());
       let client = IncrementContractClient::new(&env, &id);
   
       let mut last: Option<u32> = None;
       for _ in input.by.. {
           match client.try_increment() {
               Ok(Ok(current)) => assert!(Some(current) > last),
               Err(Ok(_)) => {} // Expected error
               Ok(Err(_)) => panic!("success with wrong type returned"),
               Err(Err(_)) => panic!("unrecognised error"),
           }
       }
   });

8. Execute the fuzz target.

   cargo +nightly fuzz run --sanitizer=thread fuzz_target_1

   info

   If you're developing on MacOS you need to add the --sanitizer=thread flag in order to work around a known issue.

This test uses the same patterns used in unit tests and integration tests:

1. Create an environment, the Env.
2. Register the contract to be tested.
3. Invoke functions using a client.
4. Assert expectations.

tip

For a full detailed example, see the fuzzing example.

info

There is another tool for fuzzing Rust code, cargo-afl. See the Rust Fuzz book for a tutorial for how to use it.

How to Get Code Coverage of Fuzz Tests

Getting code coverage data for fuzz tests requires some different tooling than when doing the same for regular Rust tests.

1. Run the fuzz tests until it has produced a corpus, just as in step 7 above.

   cargo +nightly fuzz run --sanitizer thread fuzz_target_1

2. Install the llvm-tools for the nightly compiler.

   rustup component add --toolchain nightly llvm-tools-preview

3. Run the fuzz coverage command that'll execute the corpus and write coverage data to the coverage directory in the profdata format.

   cargo +nightly fuzz coverage --sanitizer thread fuzz_target_1

4. Run the llvm-cov command to convert the profdata file to an lcov file.

   $(find $(rustc --print sysroot) -name llvm-cov) export \
       -instr-profile=fuzz/coverage/fuzz_target_1/coverage.profdata \
       -object target/$(rustc -vV | sed -n 's|host: ||p')/coverage/$(rustc -vV | sed -n 's|host: ||p')/release/fuzz_target_1 \
       --ignore-filename-regex "rustc" \
       -format=lcov \
       > lcov.info

   Load the lcov.info file into your IDE using its coverage feature. In VSCode this can be done by installing the Coverage Gutters extension and executing the Coverage Gutters: Watch command.

tip

To measure code coverage of regular Rust tests, see Code Coverage.

How to Write Property Tests

Property tests, like fuzz tests, exercise a contract with randomly generated input, but they run as ordinary #[test]s under cargo test. No nightly toolchain, no separate fuzz crate, and no special runner are required.

1. Add the proptest and proptest-arbitrary-interop crates as dev-dependencies of the contract crate.

   [dev-dependencies]
   proptest = "1"
   proptest-arbitrary-interop = "0.1"

2. Soroban contract types can only be constructed from an Env, which a proptest strategy doesn't have access to, so contract types are generated the same way as for fuzzing: through the SorobanArbitrary::Prototype pattern (see Accepting Soroban Types as Input with the SorobanArbitrary Trait). But proptest generates values from Strategys, not from Arbitrary implementations, so it cannot consume a Prototype directly. The proptest-arbitrary-interop crate's arb function bridges the gap, turning any Arbitrary type, including every SorobanArbitrary::Prototype, into a Strategy.

   The pattern mirrors the fuzz test pattern: generate the prototype with arb::<<T as SorobanArbitrary>::Prototype>(), then convert it to the real, Env-hosted contract type with .into_val(&env) inside the test body, where an Env is available.

3. Write the property test. For example, generating an Address:

   use proptest::prelude::*;
   use proptest_arbitrary_interop::arb;
   use soroban_sdk::testutils::arbitrary::SorobanArbitrary;
   use soroban_sdk::{Address, Env, IntoVal};
   
   proptest! {
       #[test]
       fn test_deposit(
           address_proto in arb::<<Address as SorobanArbitrary>::Prototype>(),
           deposit_amount in 0i128..=i128::MAX,
       ) {
           let env = Env::default();
           let address: Address = address_proto.into_val(&env);
           // call the contract with `address` and `deposit_amount`
       }
   }

   Generated Address prototypes always convert to contract addresses, never to account (G...) addresses. A property test that also needs account addresses must construct them itself with Address::from_str.

4. Run it like any other test.

   cargo test

info

Shrinking is degraded across this bridge. When a property test fails, proptest tries to shrink the failing input to a smaller, simpler one, but it does so by truncating the generated bytes and rebuilding a new value from them, rather than shrinking the value itself. As a result, the "shrunk" failing input proptest reports is generally a different input, not a smaller version of the first one that failed.

See the SorobanArbitrary proptest module for the full reference documentation, including a second example that generates a custom #[contracttype] struct.