In this section we discuss how to exercise unit, functional and performance benchmarking tests.
Audience: All users and contributors.
In CI, we execute these tests against the following build modes
- All tests on optimized and debug builds using clang
- All tests on optimized and debug builds using gcc
- All tests using address-sanitizer debug builds using the clang compiler
- All tests using memory-sanitizer debug builds using the gcc compiler
- clang-format checks
- shellcheck and shfmt checks run against shell scripts
To run a small collection of unit-tests to give you a very quick
baseline stability of the library, do:
$ make run-tests
To execute a larger set of tests, including functional and performance tests, you can do one of the following:
$ make test-results $ INCLUDE_SLOW_TESTS=true make run-tests $ INCLUDE_SLOW_TESTS=true ./test.sh
In CI, all test execution is driven by the top-level test.sh script, which exercises individual build artifacts produced for testing, as described below.
As part of the make build output the following artifacts are produced under
unit_testbinary, which runs a collection of quick-running unit tests
- A collection of stand-alone unit-test binaries in the
driver_testbinary to drive functional and performance tests
The following sections describe how to execute individual testing artifacts, for more granular test stabilization.
Running Unit tests
To run all the unit-tests, do:
Some unit-tests are designed to run with multiple threads, and larger volumes of data. These are generally named as stress tests.
You can run them standalone as follows:
See Unit testing for more details on unit test development and unit testing.
Running Functional Tests
All functional tests are executed by the
$ ./bin/driver_test --help Dispatch test --help invalid test List of tests: btree_test filter_test splinter_test log_test cache_test ycsb_test
Each functional test can be run with different parameters. Use the following syntax to get the test-specific configuration supported.
$ ./bin/driver_test <test-name> --help
$ ./bin/driver_test btree_test --help $ ./bin/driver_test splinter_test --help
test.sh script you can find
examples of the command-line parameters
that are commonly used for testing stability.
Running Performance Tests
SplinterDB performance tests are executed by
driver_test using the
option with the
Several different options are supported to execute different performance tests. These can be found using:
$ ./bin/driver_test splinter_test --help
An example usage of performance tests that are executed in our CI runs can be found here in test.sh