Unit Testing Tutorial
Contents
Introduction
If you're reading this now then I am assuming you have the Embedded ARM Development Package downloaded and unzipped. There is a UNIT_TESTING_README.txt file located in the parent directory with instructions on how to set up your environment to start unit testing the SJSU project with CGreen unit testing framework.
What is Unit Testing exactly?
Unit testing is a level of software testing where individual components or modules are tested. The purpose of unit testing is to validate that your code performs as you designed it. With the use of unit testing frameworks, stubs, and mock objects, code can be manipulated and tested without ever having to load compiled code onto a MCU.
Benefits of Unit Testing
- Increase confidence in code
- Results in modular code. Modular code is more reusable.
- Saves potential time that could be wasted debugging on a live system.
- Finds bugs before they get loaded onto a live system.
- Increases reliability of code.
- Instant visual feedback of passed/failed tests
- Regression testing - changing code either results in confidence that nothing broke or confidence that something broke and you should fix it.
Getting Started
Let's first setup our main function in test_main.cpp, which will be our foundation for future unit tests in the project :
// test_main.cpp
#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>
using namespace cgreen;
int main(int argc, char **argv)
{
TestSuite *suite = create_test_suite();
add_suite(suite, c_list_suite());
return run_test_suite(suite, create_text_reporter());
}
// create_test_suite(); only needs to be done once.
// Add your own "suite" of tests by calling add_suite() and providing a pointer to your suite.
// In this case, c_list_suite() is createdNow, in the test_c_list.c file, let's create the suite that we added to main() above.
// test_c_list.c
#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>
// The file under test
#include "c_list.h"
TestSuite *c_list_suite()
{
TestSuite *suite = create_test_suite();
add_test(suite, add_element_to_end); // Sub-test, which we will add after.
return suite;
}A Simple Test
Continuing on from our test_c_list.c file, let's perform one very basic test to verify the functionality of our C-code. We'll start by creating an Ensure() function. This is nothing more than a sub-routine test apart of the entire test suite we created. Typically, you will have an Ensure test function to test one specific functionality of your code. In this example, we only want to test the ability to add one element to the end of our linked list of c_list. The input argument to Ensure is simply a description of the test you're performing. Let's start this example by creating a pointer list.
Ensure(add_element_to_end)
{
c_list_ptr list = c_list_create();
assert_that(c_list_node_count(list), is_equal_to(1));
}Compile the program by calling make, which defaults to make all. To execute the program, simply perform ./test_all.exe
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 11ms.
code/test_c_list.c:15: Failure: c_list_suite -> add_element_to_end
Expected [c_list_node_count(list)] to [equal] [1]
actual value: [0]
expected value: [1]
Completed "c_list_suite": 1 pass, 1 failure in 9ms.
Completed "main": 1 pass, 1 failure in 20ms.Oops. Looks like we got a unit test failure because we inserted a test assert that failed. The following line assert_that(c_list_node_count(list), is_equal_to(1)); tests that the c_list list has a node list count of 1, but this failed because we didn't add any items into the c_list, thus, this did not match our expectation of having a node list of 1. Let's go ahead and add that now and see if the test passes.
Ensure(add_element_to_end)
{
c_list_ptr list = c_list_create();
c_list_insert_elm_end(list, (void*) 1);
assert_that(c_list_node_count(list), is_equal_to(1));
}Compiling and running gives:
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 14ms.
Completed "c_list_suite": 2 passes in 21ms.
Completed "main": 2 passes in 35ms.The output shows that by adding an element into the list returned a c_list_node_count of 1. We were able to test this by inserting a constraint on the assert on that specific unit test, which ultimately fails if the given inputs are not successful when ran. Here is a list of asserts and constraints you can use in your unit tests:
| Assertion | Description |
|---|---|
| assert_true(boolean) | Passes if boolean evaluates true |
| assert_false(boolean) | Fails if boolean evaluates true |
| assert_equal(first, second) | Passes if 'first == second' |
| assert_not_equal(first, second) | Passes if 'first != second' |
| assert_string_equal(char *, char *) | Uses 'strcmp()' and passes if the strings are equal |
| assert_string_not_equal(char *, char *) | Uses 'strcmp()' and fails if the strings are equal |
Each assertion has a default message comparing the two values, but these can be substituted with your own failure messages, then you must use the *_with_message() counterparts...
| Assertion |
|---|
| assert_true_with_message(boolean, message, …) |
| assert_false_with_message(boolean, message, …) |
| assert_equal_with_message(tried, expected, message, …) |
| assert_not_equal_with_message(tried, unexpected, message, …) |
| assert_string_equal_with_message(char *, char *, message, …) |
| assert_string_not_equal_with_message(char *, char *, message, …) |
| Constraint | Passes if actual value/expression… |
|---|---|
| is_true | evaluates to true |
| is_false | evaluates to false |
| is_null | equals null |
| is_non_null | is a non null value |
| is_equal_to(value) | '== value' |
| is_not_equal_to(value) | '!= value' |
| is_greater_than(value) | '> value' |
| is_less_than(value) | '< value' |
| is_equal_to_contents_of(pointer, size) | matches the data pointed to by pointer to a size of size bytes |
| is_not_equal_to_contents_of(pointer, size) | does not match the data pointed to by pointer to a size of size bytes |
| is_equal_to_string(value) | are equal when compared using strcmp() |
| is_not_equal_to_string(value) | are not equal when compared using strcmp() |
| contains_string(value) | contains value when evaluated using strstr() |
| does_not_contain_string(value) | does not contain value when evaluated using strstr() |
| is_equal_to_double(value) | are equal to value within the number of significant digits (which you can set with a call to significant_figures_for_assert_double_are(int figures)) |
| is_not_equal_to_double(value) | are not equal to value within the number of significant digits |
| is_less_than_double(value) | < value withing the number of significant digits |
| is_greater_than_double(value) | > value withing the number of significant digits |
Another Unit Test (C++)
The first example showed a C function unit test. The following will serve as an example of how to test C++, or primarily classes. Let's skip forward to the implementation of our simple foo.cpp, foo.hpp, turtle.hpp, and turtle.cpp files in the next sample unit test. We have a Foo class, which has a method called add that creates a local Turtle class object, calls draw and adds it to the input argument x. The Turtle class method draw returns a constant value of 10 unconditionally.
// foo.hpp & foo.cpp - Foo class that makes a call to a Turtle class
class Foo
{
public:
int add(int x);
};
int Foo::add(int x)
{
Turtle turtle;
return x + turtle.turtle_draw();
}
// End foo.hpp & foo.cpp
// turtle.hpp & turtle.cpp - Random turtle class
class Turtle {
public:
int turtle_draw(void);
};
int Turtle::turtle_draw(void)
{
return 10;
}
// End turtle.hpp & turtle.cppNext, let's skip forward to the test_foo.cpp, where we will be testing the Foo module source file.
#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>
using namespace cgreen;
// The file under test
#include "foo.hpp"
// Other files we need to include
#include "turtle.hpp"
Ensure(testing_call_to_turtle)
{
Foo foo;
assert_that(foo.add(1), is_equal_to(21));
}
TestSuite *foo_suite()
{
TestSuite *suite = create_test_suite();
add_test(suite, testing_call_to_turtle);
return suite;
}As you can see we have a unit test called testing_call_to_turtle, where we will be testing the invocation of Foo.add(). The function add() makes an function call to another public method that is not located in the source file, Foo. Let's compile and execute the program now to analyze what happens under a normal scenario.
/cygdrive/c/SJSU_Dev/projects/cgreen_test/example/foo.cpp:9: undefined reference to `Turtle::turtle_draw()'It looks like we got a compiler error. This means the unit test framework is working as expected. We need to set up the stub or mock for any external functions that this foo.cpp file calls. Adding in the function declaration with a return of the mock() function will allow us to do just so in this CGreen unit test framework.
int Turtle::turtle_draw(void) { return mock(); }
Ensure(testing_call_to_turtle)
{
Foo foo;
}Let's pause to take a break and go over what a mock is:
- Mock - a mock is a programmable object. In C objects are limited to function, and in C++ mocks are basically classes. The macro mock() compares the incoming parameters with any expected values and dispatches messages to the test suite if there is a mismatch. It also returns any values that have been preprogrammed in the test.
Let's go ahead and setup our expectations of any calls foo.draw will make. In this case, it calls turtle.draw so we have to set up 1 expect of turtle.draw for each foo.add performed.
int Turtle::turtle_draw(void) { return mock(); }
Ensure(testing_call_to_turtle)
{
Foo foo;
// Expect the mock object, but return a value of 20. This overwrites the original implementation of turtle_draw
expect(turtle_draw, will_return(20));
// Test that when add(1) is called, the value 21 is returned based on our expect() we just made
assert_that(foo.add(1), is_equal_to(21));
}This yields a successful output of all tests passing :
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 14ms.
Completed "c_list_suite": 2 passes in 11ms.
Completed "main": 2 passes in 25ms.Now, what happens if we change the return value of our expected mock object?
int Turtle::turtle_draw(void) { return mock(); }
Ensure(testing_call_to_turtle)
{
Foo foo;
expect(turtle_draw, will_return(100));
assert_that(foo.add(1), is_equal_to(21));
}$ ./test_all.exe
Running "main" (2 tests)...
code/test_foo.cpp:25: Failure: foo_suite -> testing_call_to_turtle
Expected [foo.add(1)] to [equal] [21]
actual value: [101]
expected value: [21]
Completed "foo_suite": 1 failure in 14ms.
Completed "c_list_suite": 1 pass, 1 failure in 20ms.
Completed "main": 1 pass, 1 failure in 34ms.As expected our test failed because the assertion result did not match because the return value was changed. This example demonstrates the fact that you can manipulate any external libraries or code that your source code depends on and you can test that it performs the way you expect it should.
