Mbed SDK cpputest usage
Mbed OS 2 and Mbed OS 5
This is a guide for Mbed OS 2. If you’re working with Mbed OS 5, please see the Testing guide in the new handbook.
Introduction - CppUTest in Mbed SDK testing¶
CppUTest library¶
CppUTest (http://cpputest.github.io/) is a C /C++ based unit xUnit test framework for unit testing and for test-driving your code. It is written in C++ but is used in C and C++ projects and frequently used in embedded systems but it works for any C/C++ project.
Mbed SDK test suite supports writing tests using CppUTest. All you need to do it to provide CppUTest sources and includes with Mbed SDK port. This is already done for you so all you need to do it to get proper sources in your project directory.
CppUTest’s core design principles are:
- Simple in design and simple in use.
- Portable to old and new platforms.
- Build with Test-driven Development in mind
From where you can get more help about CppUTest library and unit testing¶
- To read CppUTest manual please go here: http://cpputest.github.io/manual.html
- CppUTest forum: https://groups.google.com/forum/?fromgroups#!forum/cpputest
- CppUTest webpage on GitHub: https://github.com/cpputest/cpputest
- Finally, if you think unit testing is new concept for you, you can have a grasp of it on Wikipedia pages: http://en.wikipedia.org/wiki/Unit_testing and continue from there.
How to add CppUTest to your current Mbed SDK installation¶
Do I need CppUTest port for Mbed SDK?¶
Yes, you do. If you want to use CppUTest with Mbed SDK you need to have CppUTest version with ARMCC compiler (only ARM flavor for now) port and Mbed SDK console port (if you want to have output on serial port). All is already prepared by Mbed engineers and you can get it for example here: http://mbed.org/users/rgrover1/code/CppUTest/
Prerequisites¶
- Installed git client e.g. http://git-scm.com/downloads
- Installed Mercurial client e.g. http://tortoisehg.bitbucket.org/
How to install¶
We want to create directory structure similar to one below:
Your Project
│
├───cpputest
│ ├───include
│ └───src
└───mbed
├───libraries
├───travis
└───workspace_tools
Please go to directory with your project. For example it could be c:\Projects\Project.
cd c:\Projects\Project
If your project directory already has your Mbed SDK repository included just execute below command. It should download CppUTest with Mbed SDK port.
hg clone https://mbed.org/users/rgrover1/code/cpputest/
You should see something like this after you execute Mercurial clone command:
c:\Projects\Project>hg clone https://mbed.org/users/rgrover1/code/cpputest/ destination directory: cpputest requesting all changes adding changesets adding manifests adding file changes added 3 changesets with 69 changes to 42 files updating to branch default 41 files updated, 0 files merged, 0 files removed, 0 files unresolved
Confirm your project structure. It should look more or less like this:
c:\Projects\Project>ls cpputest mbed
From now on CppUTest is in correct path. Each time you want to compile unit tests for CppUTest build script will always look for CppUTest library in the same directory where mbed library is.
New off-line Mbed SDK project with CppUTest support¶
If you are creating new Mbed SDK project and you want to use CppUTest with it you need to download bot Mbed SDK and CppUTest with mbed port to the same directory. You can do it like this:
cd c:\Projects\Project
git clone https://github.com/mbedmicro/mbed.git
hg clone https://mbed.org/users/rgrover1/code/cpputest/
It should look more or less like on screen below:
After above three steps you should have proper directory structure. All you need to do now is to configure your private_settings.py in mbed/workspace_tools/ directory. Please refer to Mbed SDK build script environment's Configure workspace tools to work with your compilers Wiki page chapter for details about how to configure mbed SDK so it's aware of your supported toolchains.
CppUTest port¶
To make sure you actualy have CppUTest library with Mbed SDK port you can go to cpputest ARMCC platform directory:
cd c:\Projects\Project\cpputest\src\Platforms\armcc\
And open file UtestPlatform.cpp.
You should find part of code responsible for porting console on default serial port of the mbed device:
UtestPlatform.cpp
#include "Serial.h"
using namespace mbed;
int PlatformSpecificPutchar(int c)
{
/* Please modify this block for test results to be reported as
* console output. The following is a sample implementation using a
* Serial object connected to the console. */
#define NEED_TEST_REPORT_AS_CONSOLE_OUTPUT 1
#if NEED_TEST_REPORT_AS_CONSOLE_OUTPUT
extern Serial console;
#define NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE 1
#if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE
/* CppUTest emits \n line terminators in its reports; some terminals
* need the linefeed (\r) in addition. */
if (c == '\n') {
console.putc('\r');
}
#endif /* #if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE */
return (console.putc(c));
#else /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
return (0);
#endif /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
}
You can find cpputest UT test runner main function in mbed sources;
c:\Projects\Project\mbed\libraries\tests\utest\testrunner\testrunner.cpp
Test runner code only defined console object and executes all unit tests.
testrunner.cpp
#include "CppUTest\CommandLineTestRunner.h"
#include <stdio.h>
#include "mbed.h"
#include "testrunner.h"
#include "test_env.h"
/**
Object 'console' is used to show prints on console.
It is declared in \cpputest\src\Platforms\armcc\UtestPlatform.cpp
*/
Serial console(STDIO_UART_TX, STDIO_UART_RX);
int main(int ac, char** av)
{
unsigned failureCount = 0;
{
// Some compilers may not pass ac, av so we need to supply them ourselves
int ac = 2;
char* av[] = {__FILE__, "-v"};
failureCount = CommandLineTestRunner::RunAllTests(ac, av);
}
notify_completion(failureCount == 0);
return failureCount;
}
Unit test location¶
Unit tests' source code is located in below directory:
c:\Projects\Project\mbed\libraries\tests\utest
Each sub directory except testrunner contains compilable unit test source file(s) with test groups and test cases. You can see utest structure below. Please note this is just example and in the future this directory will contain many sub directories with unit tests.
c:\Projects\Project\mbed\libraries\tests\utest>tree utest ├───basic ├───semihost_fs └───testrunner
Define unit tests in test suite¶
All tests defined in test suite are described in mbed/workspace_tools/tests.py file. This file stores data structure TESTS which is a list of simple structures describing each test. Below you can find example of TESTS structure which is configuring one of the unit tests (defines which ).
mbed\\workspace_tools\tests.py
{
"id": "UT_2", "description": "Semihost file system",
"source_dir": join(TEST_DIR, "utest", "file"),
"dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
"automated": False,
"mcu": ["LPC1768", "LPC2368", "LPC11U24"]
},
Note: In dependency section we've added library CPPUTEST_LIBRARY which is pointing build script to CppUTest library with mbed port. This is a must for unit tests to be compiled with CppUTest library.
Tests are now divided into two types:
'Hello world' tests¶
First we call 'hello world' tests. They do not dependent on CppUTest library and are monolithic programs usually composed of one main function. Yo can find this tests in below example directories:
- mbed/libraries/tests/mbed/
- mbed/libraries/tests/net/
- mbed/libraries/tests/rtos/
- mbed/libraries/tests/usb/
Unit test tests¶
Second group of tests are unit tests. They are using CppUTest library and require you to write TEST_GROUPs and TESTs in your test files. Test suite will add test runner sources to your test automatically so you can concentrate on writing tests.
Example of unit test¶
mbed\libraries\tests\utest\semihost_fs\semihost_fs.cpp
#include "CppUTest/TestHarness.h"
#include "mbed.h"
#include "semihost_api.h"
#include <stdio.h>
#define FILENAME "/local/out.txt"
#define TEST_STRING "Hello World!"
TEST_GROUP(FirstTestGroup)
{
FILE *test_open(const char *mode) {
FILE *f = fopen(FILENAME, mode);
return f;
}
bool test_write(FILE *f, char *str, int str_len) {
int n = fprintf(f, str);
return (n == str_len) ? true : false;
}
bool test_read(FILE *f, char *str, int str_len) {
int n = fread(str, sizeof(unsigned char), str_len, f);
return (n == str_len) ? true : false;
}
bool test_close(FILE *f) {
int rc = fclose(f);
return rc ? true : false;
}
};
TEST(FirstTestGroup, FirstTest)
{
CHECK_TEXT(semihost_connected(), "Semihost not connected")
LocalFileSystem local("local");
char *str = TEST_STRING;
char *buffer = (char *)malloc(sizeof(unsigned char) * strlen(TEST_STRING));
int str_len = strlen(TEST_STRING);
CHECK_TEXT(buffer != NULL, "Buffer allocation failed");
CHECK_TEXT(str_len > 0, "Test string is empty (len <= 0)");
{
// Perform write / read tests
FILE *f = NULL;
// Write
f = test_open("w");
CHECK_TEXT(f != NULL, "Error opening file for writing")
CHECK_TEXT(test_write(f, str, str_len), "Error writing file");
CHECK_TEXT(test_close(f) != EOF, "Error closing file after write");
// Read
f = test_open("r");
CHECK_TEXT(f != NULL, "Error opening file for reading")
CHECK_TEXT(test_read(f, buffer, str_len), "Error reading file");
CHECK_TEXT(test_close(f) != EOF, "Error closing file after read");
}
CHECK(strncmp(buffer, str, str_len) == 0);
}
Example of 'hello world' test¶
mbed\libraries\tests\mbed\i2c_MMA8451Q\main.cpp
#include "mbed.h"
#include "MMA8451Q.h"
#include "test_env.h"
#ifdef TARGET_KL05Z
#define SDA PTB4
#define SCL PTB3
#else
#define SDA PTE25
#define SCL PTE24
#endif
namespace {
const int MMA8451_I2C_ADDRESS = 0x1D << 1; // I2C bus address
const float MMA8451_DIGITAL_SENSITIVITY = 4096.0; // Counts/g
}
float calc_3d_vector_len(float x, float y, float z) {
return sqrt(x*x + y*y + z*z);
}
#define TEST_ITERATIONS 25
#define TEST_ITERATIONS_SKIP 5
#define MEASURE_DEVIATION_TOLERANCE 0.025 // 2.5%
int main(void) {
DigitalOut led(LED_GREEN);
MMA8451Q acc(SDA, SCL, MMA8451_I2C_ADDRESS);
bool result = true;
printf("WHO AM I: 0x%2X\r\n\n", acc.getWhoAmI());
for (int i = 0; i < TEST_ITERATIONS; i++) {
if (i < TEST_ITERATIONS_SKIP) {
// Skip first 5 measurements
continue;
}
const float g_vect_len = calc_3d_vector_len(acc.getAccX(), acc.getAccY(), acc.getAccZ()) / MMA8451_DIGITAL_SENSITIVITY;
const float deviation = fabs(g_vect_len - 1.0);
const char *succes_str = deviation <= MEASURE_DEVIATION_TOLERANCE ? "[OK]" : "[FAIL]";
result = result && (deviation <= MEASURE_DEVIATION_TOLERANCE);
printf("X:% 6d Y:% 6d Z:% 5d GF:%0.3fg, dev:%0.3f ... %s\r\n", acc.getAccX(), acc.getAccY(), acc.getAccZ(), g_vect_len, deviation, succes_str);
wait(0.5);
led = !led;
}
notify_completion(result);
}
Example¶
In below example we will run two example unit tests that are now available. tests UT_1 and UT_2 are unit tests used for now only to check if mbed SDK works with CppUTest library and if tests are being executed. In future number of unit tests will increase, nothing is also should stop you from writing and executing your own unit tests!
Example configuration¶
By default unit tests UT_1 and UT_2 are not automated - simply test suite will ignore them. Also we do not want to create dependency to CppUTest library each time someone executes automation.
Note: To compile UT_1 and UT_2 tests CppUTest library described above installation is needed and not all users wish to add UT libs to their project. Also new to mbed users may find it difficult. This is why unit testing is an extra feature active only after you deliberately install and enable needed components.
Bellow snippet shows how to modify 'automated' flag so test suite will consider unit tests UT_1 and UT_2 as part of "automated test portfolio". Just change flag 'automated' from False to True.
workspace_tools\tests.py
# CPPUTEST Library provides Unit testing Framework
#
# To write TESTs and TEST_GROUPs please add CPPUTEST_LIBRARY to 'dependencies'
#
# This will also include:
# 1. test runner - main function with call to CommandLineTestRunner::RunAllTests(ac, av)
# 2. Serial console object to print test result on serial port console
#
# Unit testing with cpputest library
{
"id": "UT_1", "description": "Basic",
"source_dir": join(TEST_DIR, "utest", "basic"),
"dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
"automated": True,
},
{
"id": "UT_2", "description": "Semihost file system",
"source_dir": join(TEST_DIR, "utest", "semihost_fs"),
"dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
"automated": True,
"mcu": ["LPC1768", "LPC2368", "LPC11U24"]
},
Execute tests¶
In my test I will use common NXP LPC1768 mbed enabled board because unit test UT_2 is checking semi-host functionality which is available on this board and handful of others.
Configure your test_spec.json and muts_all.json files and set mbed disk and serial port.
singletest.py -i test_spec.json -M muts_all.json -n UT_1,UT_2 -V
Building library CMSIS (LPC1768, ARM)
Building library MBED (LPC1768, ARM)
Building library CPPUTEST (LPC1768, ARM)
Building project BASIC (LPC1768, ARM)
Executing 'python host_test.py -p COM77 -d E:\ -t 10'
Test::Output::Start
Host test instrumentation on port: "COM77" and disk: "E:\"
TEST(FirstTestGroup, FirstTest) - 0 ms
OK (1 tests, 1 ran, 3 checks, 0 ignored, 0 filtered out, 3 ms)
{{success}}
{{end}}
Test::Output::Finish
TargetTest::LPC1768::ARM::UT_1::Basic [OK] in 2.43 of 10 sec
Building library CPPUTEST (LPC1768, ARM)
Building project SEMIHOST_FS (LPC1768, ARM)
Executing 'python host_test.py -p COM77 -d E:\ -t 10'
Test::Output::Start
Host test instrumentation on port: "COM77" and disk: "E:\"
TEST(FirstTestGroup, FirstTest) - 9 ms
OK (1 tests, 1 ran, 10 checks, 0 ignored, 0 filtered out, 10 ms)
{{success}}
{{end}}
Test::Output::Finish
TargetTest::LPC1768::ARM::UT_2::Semihost file system [OK] in 2.43 of 10 sec
Test summary:
+--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
| Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
| OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
| OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
+--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
Result: 2 OK
Completed in 12.02 sec
You can compile unit tests using various number of supported compilers, below just few examples with working solutions:
Test summary: +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+ | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops | +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+ | OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 | | OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 | | OK | LPC1768 | uARM | UT_1 | Basic | 2.43 | 10 | 1/1 | | OK | LPC1768 | uARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 | | OK | LPC1768 | GCC_ARM | UT_1 | Basic | 2.43 | 10 | 1/1 | | OK | LPC1768 | GCC_ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 | | OK | LPC1768 | GCC_CR | UT_1 | Basic | 3.44 | 10 | 1/1 | | OK | LPC1768 | GCC_CR | UT_2 | Semihost file system | 3.43 | 10 | 1/1 | +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+ Result: 8 OK Completed in 55.85 sec