Unit Testing

Unit testing introduction

Tests are a great way to validate the operation of software and should be a part of any commercial or serious project. electricui-embedded tests to prevent functionality regressions, catch bugs and let us sleep at night. Some other tools mentioned in this section are more than just unit tests, linting checks and automated fuzzing tests help find potential issues outside the test corpus.

The electricui-embedded/test directory contains the testing related code and helper scripts. We assume the use of a unix system, and bash-like shell.

We use Unity for tests, along with CMock as part of the Ceedling test suite. This is a Ruby/Rake based system which provides a series of helper utilities for test management, automatic mock creation, and other assistive tools like easy code coverage reports.

Install and preparation

We don't include Ceedling's vendor files in the repo, so first runs need to 're-initalise' the test structure.

1. The test/setup_tests.sh script helps get started really quickly, helping you get the ceedling gem installed, and re-initalising the project.
   • You might need to add execution permission first. From /test, run chmod +x setup_tests.sh.
   • If you don't have ceedling installed, it will prompt to install the ruby gem.
2. The script will create the relevant files and automatically reject changes to our existing configuration files.
3. Once setup is complete, just run ceedling or ceedling test:all, results will be printed to the console.

Adding & Modifying tests

The tests are in a series of files in the tests/test/ directory and should provide enough of an example. Ceedling's vendor folder has docs for Unity, CMock and Ceedling if you need more detailed guides and examples.

Code Coverage Reports

Ceedling includes the helper for this, providing gcov is available on your system.

ceedling gcov:all to generate the coverage map, then ceedling utils:gcov to generate the HTML formatted report which provides line and branch level coverage details.

Lint Analysis

oclint is used to provide a basic level of static analysis, aimed at identifying code smells and syntactical errors which might cause undesired behaviour or needlessly complex source code.

1. Its available in binary format or as sources from the website, or you can use your package manager if its available (brew install oclint) etc.
2. Run ./lint_checks.sh to run oclint with the library sources. The script applies some small tweaks to the linting ruleset and enables the static analysis functionality.
3. Results are printed to the console.

Small numbers of P2 and P3 errors are acceptable in certain situations, as the cyclomatic complexity and function length guides are difficult to overcome in some situations without needlessly obscuring functionality behind denser syntax.

The linting tool is best used as a guide or warning while developing, and many low severity warnings don't necessarily require action.

Fuzzing

American Fuzzy Lop, afl-fuzz is used to generate mutated input data and pass it to the decoder. This allows us to test the effects of potentially underhanded input messages, though this test program is simplistic and slightly naive.

1. Install the afl suite by following the install instructions or using your package manager (brew install afl-fuzz on OSX, pacman -S afl for Arch, etc).
2. Modify the makefile to correctly point to the afl-gcc (most *nix systems) or afl-clang (OSX) build tool to suit your system.
3. From the test/fuzz directory, call make all to build an instrumented binary and begin testing.
4. Wait a while while it attempts various modifications and mutations from the input files.