RustBrock/Test_Organization.md

Test Organization

The Rust community thinks about tests in terms of two main categories

• Unit tests
• Integration Tests

Unit tests are small and more focused, testing one module in isolation at a time and can test private interfaces

Integration tests are entirely external to your library and use your code in the same way any other external code would. Using only the public interface and potentially exercising multiple modules per test.

Unit Tests

The purpose of these kinds of tests is to test each unit of code in isolation from the rest of the code to pinpoint where the code is and isn't working as expected

You put unit tests in the src directory in each file the code that they are testing.

The convention is to create a module named tests in each file to contain the test functions and to annotate the module with cfg(test)

The Tests Module and #[cfg(test)]

The #[cfg(test)] annotation on the tests module tells Rust to compile and run the test code only when you run cargo test, not when you run cargo build.

This both saves time when compiling when you only want to build the library and saves space in the resultant compiled artifact because the tests are not included.

You will see that because integration tests go in a different directory, they don't need the #[cfg(test)] annotation.

However because unit tests go in the same files as the code you will use the #[cfg(test)] annotation to specify that they shouldn't be included in the compiled result

Recall when we generated the new adder project. Cargo generated this code for us:

pub fn add(left: usize, right: usize) -> usize {
    left + right
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn it_works() {
        let result = add(2, 2);
        assert_eq!(result, 4);
    }
}

On the auto generated tests module the attribute cfg stands for configuration and tells Rust that the following item should only be included given a certain config option

In this case the config option is test which is provided by Rust for compiling and running the tests.

Using the cfg attribute, Cargo compiles our test code only if we actively run the tests with cargo test

This includes any helper functions that might be within this module, and includes functions annotated with #[test]

Testing Private Functions

It is debated about whether or not private functions should be tested directly and other languages make it difficult or impossible to do so.

Regardless of which testing ideology you believe, Rust's privacy rules do allow you to test private functions

Consider this code

pub fn add_two(a: usize) -> usize {
    internal_adder(a, 2)
}

fn internal_adder(left: usize, right: usize) -> usize {
    left + right
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn internal() {
        let result = internal_adder(2, 2);
        assert_eq!(result, 4);
    }
}

Note that internal_adder function is not marked as pub.

Tests are just Rust code, and the tests module is just another module.

Because the module is a child module it can use the items in their ancestor modules.

In this test we bring all of the tests module's parent items into the scope using the use super::* and then we can call the private function internal_adder

If you don't believe that internal functions should be tested there is nothing in Rust that will compel you to do so.

Integration Tests

In Rust integration tests are entirely external to library.

They use your library in the same way any any external code would, which means that they can only call functions that are part of your library's public API.

The purpose of these is to tests weather units work together correctly.

Units of code that work correctly on their own could have problems when put together, so test coverage of the integrated code is important as well.

To create integration tests you need a tests directory

The tests Directory

We create a tests directory at the top level of our project directory next to src

Cargo knows how to look for integration test files in this directory

We can make as many as we want and Cargo will compile each of the files as an individual crate

Here is how it should be setup and what your file directory system should look like

adder
├── Cargo.lock
├── Cargo.toml
├── src
│   └── lib.rs
└── tests
    └── integration_test.rs

Here is what tests/integration_test.rs should contain

use adder::add_two;

#[test]
fn it_adds_two() {
    let result = add_two(2);
    assert_eq!(result, 4);
}

Each file in the tests directory is a separate crate so we need to bring our library into each test crat's scope

So we need to use use adder::add_two; in this case. We don't need unit tests because we aren't writing tests for them.

We don't need to annotate any code in tests/integration_test.rs with #[cfg(test)].

Cargo treats the tests directory specially and compiles files in this directly only when we run cargo test

Here would be the result output in this case

$ cargo test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.31s
     Running unittests src/lib.rs (target/debug/deps/adder-1082c4b063a8fbe6)

running 1 test
test tests::internal ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/integration_test.rs (target/debug/deps/integration_test-1082c4b063a8fbe6)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests adder

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

There are three sections of the output

1. The Unit Tests
2. The Integration test
3. The Doc Tests

Note that if any test in a section fails the following sections will not run For example if a unit test fails, there wont be any output for integration and doc tests because those tests will only be run if all unit tests are passing.

The integration tests scetion starts after the unit tests (one per line) and the unit summary test section.

It beings with th line Running tests/integration_test.rs

Next there is a line for each test function in that integration tests and a summary line of the integration test just before the Doc-tests adder section starts

Each integration test file has its own section, so if more files in the test directory, there will be more integration test sections.

We can run a particular integration test function by specifying the test function's name as an argument to cargo test

To run all the tests in a particular integraion file use the --test argument of the cargo test followed by the name of the file

Here is an example of running a particular file

$ cargo test --test integration_test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.64s
     Running tests/integration_test.rs (target/debug/deps/integration_test-82e7799c1bc62298)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Submodules in Integration Tests

As you add more integartion tests, you might want to make more files in the tests directory to help organize them

For example you might want to group the test functions by the functionality that they are testing

As mentioned previously each file in the test dir is compiled as its own separate crate which is useful for creating separate scopes to more closely imitate the way end users will be using your crate.

This means files in the tests directory don't share the same behavior as files in the src do

A refesher on speerate code modules can be found here

The different behavior of tests directroy files is most present when you have a set of helper fnctions to use in multople integration test files.

For this you can follow the steps in the "Separating Modules in Different Files" to extract them into a common module.

For example if we create a file in tests/common.rs and place a function named setup in it.

We could add some code to setup that we want to call from multiple test function in multiple test files

pub fn setup() {
    // setup code specific to your library's tests would go here
}

When we run cargo test we see a new section for the output for the common.rs file even though this tet doent contain any test functions nor did we call the setup function from anywhere

Here is the output

$ cargo test
   Compiling adder v0.1.0 (file:///projects/adder)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.89s
     Running unittests src/lib.rs (target/debug/deps/adder-92948b65e88960b4)

running 1 test
test tests::internal ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/common.rs (target/debug/deps/common-92948b65e88960b4)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/integration_test.rs (target/debug/deps/integration_test-92948b65e88960b4)

running 1 test
test it_adds_two ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests adder

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Having common appear in the test results with running 0 tests displayed for it is not what we intended.

We just want to share some code with the other integration test files.

To avoid having common appear in the test outputs you would create a file in tests/common/mod.rs instead of tests/common.rs

Here is how the project directory should look like

├── Cargo.lock
├── Cargo.toml
├── src
│   └── lib.rs
└── tests
    ├── common
    │   └── mod.rs
    └── integration_test.rs

This is the older naming convention that Rst also understands for adding different modules.

Naming the file this way tells Rust to not treat the common module as an integration test file

When we move the setup function code into mod.rs and delete the common.rs file the section in the test output will no longer appear.

file in subdirectories of the tests directory don't get compiled as separate crates or have sections in the test output.

We can use the mod.rs file from any integration test file as a module

Here is an example of calling the setup function form the it_adds_two test in tests/integration_test.rs

use adder::add_two;

mod common;

#[test]
fn it_adds_two() {
    common::setup();

    let result = add_two(2);
    assert_eq!(result, 4);
}

Integration Test for Binary Crates

If your project is a binary crate that only contains a src/main.rs file and doesn't have a src/lib.rs file we can't create integration tests in the tests directory and bring functions defined in that main.r file into scope with a use statement.

Only library crates expose functions that other crates can use.

Binary crates are meant to run on thier own.

This is why Rust projects have a very simple main.rs file that calls upon logic that lives in the library crate

Using that structure integration tests can test the library crate using use to make the important functionality available.

If the important funtionality works the small amount of code in src/main.rs file will work as well and that small amount of code doesn't need to be tested.