|
|
|
# Testing
|
|
|
|
## Catch Test Framework
|
|
|
|
A powerful tool to facilitate writing test for our software is the [Catch Test Framework](https://github.com/catchorg/Catch2/tree/Catch1.x). In this page I will provide basic examples on how to use it in our codebase, but it is strongly recommended that you read the [tutorial on the Catch Github repo](https://github.com/catchorg/Catch2/blob/Catch1.x/docs/tutorial.md), as well as the [Reference](https://github.com/catchorg/Catch2/blob/Catch1.x/docs/Readme.md).
|
|
|
|
Please note that the version of Catch used in our codebase is Catch1.x.
|
|
|
|
|
|
|
|
## Testing strategy (WIP)
|
|
|
|
|
|
|
|
## Writing tests
|
|
|
|
### Test source file
|
|
|
|
Tests that use catch must be located under the src/test/catch1 directory in skyward-boardcore.
|
|
|
|
Source names must start with `test` and words-must-be-separated-by-dashes. Example: `test-temperature-sensor.cpp`.
|
|
|
|
Once you have created your source file, the first thing is to add the following line of code:
|
|
|
|
```c
|
|
|
|
#ifdef STANDALONE_CATCH1_TEST
|
|
|
|
#include "catch1-tests-entry.cpp"
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#include <catch.hpp>
|
|
|
|
```
|
|
|
|
The first 3 lines are needed in the case you want to run the test by itself, defining its own entrypoint. More on this in the next section.
|
|
|
|
Once you've added this 4 lines of code, you are ready to write your tests. Take a look at the examples in the Catch Github repo linked above, and at the examples in `src/tests/examples` in skyward-boardcore to know how to proceed.
|
|
|
|
### Entrypoint configuration
|
|
|
|
To run the tests, you first need to define the entrypoint in `sbs.conf`. You will find two types of entrypoints for a test:
|
|
|
|
#### Standalone test
|
|
|
|
This is a test that will run alone. Useful when you are writing the test and don't want to wait for all other tests to run together with the one you are writing.
|
|
|
|
The entrypoint for the is defined as follows:
|
|
|
|
```tcl
|
|
|
|
[example-test-factorial] #name of the entrypoint
|
|
|
|
Type: test
|
|
|
|
#The board you want to run the test on
|
|
|
|
BoardId: stm32f429zi_stm32f4discovery
|
|
|
|
#Name of the binary, must be the same as the name of the entrypoint
|
|
|
|
BinName: example-test-factorial
|
|
|
|
Include: #Required sources, if any
|
|
|
|
Defines: -DSTANDALONE_CATCH1_TEST
|
|
|
|
#Relative path of the .cpp file, without extension
|
|
|
|
Main: examples/example-test-factorial
|
|
|
|
```
|
|
|
|
Note that, in order to run the test by itself, we need to define `STANDALONE_CATCH1_TEST`.
|
|
|
|
#### Grouped tests
|
|
|
|
Before committing changes, especially to the *master* branch, we want to be sure that no problems are present in the code. We could run all the tests one by one, but compiling, flashing and executing them one by one would be a nightmare as the number of tests increase. Fortunately, the Catch framework allows for tests defined in different source files to run together with little to no effort required by the developer.
|
|
|
|
In fact, in order to do so, we just to need to compile the test source files together with the `catch1-tests-entry.cpp` entrypoint source.
|
|
|
|
The configuration in `sbs.conf` will look like this:
|
|
|
|
```tcl
|
|
|
|
#Add here all the source files of the tests you want to run
|
|
|
|
[example-tests]
|
|
|
|
Type: srcfiles
|
|
|
|
Files: src/tests/examples/example-test-factorial
|
|
|
|
src/tests/examples/example-test-fsm
|
|
|
|
|
|
|
|
|
|
|
|
# Run all example tests togheter
|
|
|
|
[example-catch1-tests]
|
|
|
|
Type: test
|
|
|
|
BoardId: stm32f429zi_stm32f4discovery
|
|
|
|
BinName: example-catch1-tests
|
|
|
|
#Add all required sources + tests sources
|
|
|
|
Include: %shared %example-tests
|
|
|
|
Defines:
|
|
|
|
Main: catch1-tests-entry
|
|
|
|
```
|
|
|
|
When you run `example-catch1-tests`, both the tests in `example-test-fsm.cpp` and `example-test-factorial.cpp` will run.
|
|
|
|
|
|
|
|
|
|
|
|
#### Test execution configuration
|
|
|
|
*Catch* provides some command line options to configure the test output and decide which tests to run. The options are described in the [command line reference page](https://github.com/catchorg/Catch2/blob/Catch1.x/docs/command-line.md) on the Catch Github repo.
|
|
|
|
In boardcore, these command line options are passed via a *define* in sbs.conf: `CATCH1_CL_OPTIONS`.
|
|
|
|
Usage: `-DCATCH1_CL_OPTIONS="\"[tag1][tag2] -a -b -c\""`
|
|
|
|
Example:
|
|
|
|
```tcl
|
|
|
|
[example-catch1-tests]
|
|
|
|
Type: test
|
|
|
|
BoardId: stm32f429zi_stm32f4discovery
|
|
|
|
BinName: example-catch1-tests
|
|
|
|
#Add all required sources + tests sources
|
|
|
|
Include: %shared %example-tests
|
|
|
|
Defines: -DCATCH1_CL_OPTIONS="\"[homeone] -s\""
|
|
|
|
Main: catch1-tests-entry
|
|
|
|
``` |
|
|
\ No newline at end of file |
