View source for Unit Testing Tutorial

==== Introduction ====
This tutorial contains a few examples and a walk-through of how to start unit testing the SJSU project, which utilizes the CGreen unit testing framework.  I am assuming you have the Embedded ARM Development Package downloaded and unzipped. There is a UNIT_TESTING_README.txt file located in the parent directory with instructions on how to set up your environment to start unit testing the SJSU project with CGreen unit testing framework.

==== What is Unit Testing exactly? ====
Unit testing is a level of software testing where individual components or modules are tested. The purpose of unit testing is to validate that your code performs as you designed it. With the use of unit testing frameworks, stubs, and mock objects, code can be manipulated and tested without ever having to load compiled code onto a MCU. This statement is by no means saying that you shouldn't system test your code, but rather a statement of the power of using unit tests in your development life-cycle.

==== Benefits of Unit Testing ====
* Increase confidence in code
* Results in modular code. Modular code is more reusable.
* Saves potential time that could be wasted debugging on a live system.
* Finds bugs before they get loaded onto a live system.
* Increases reliability of code.
* Instant visual feedback of passed/failed tests
* Regression testing - changing code either results in confidence that nothing broke or confidence that something broke and you should fix it.

==== Why CGreen? ====
Cgreen is a modern unit test and mocking framework for C and C++. It's fast to build, the code is clean, and the API is fluent and expressive with the same modern syntax across C and C++. Each test is isolated, which is great because it prevents intermittent failures and cross-test dependencies. The Cgreen frame provides built-in mocking for C and is compatible with other C++ mocking libraries.

==== Getting Started ====
Let's first setup our main function in '''''test_main.cpp''''', which will be our foundation for future unit tests in the project :

<syntaxhighlight lang="c">
// test_main.cpp

#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>
using namespace cgreen;

int main(int argc, char **argv) 
{
    TestSuite *suite = create_test_suite();
    add_suite(suite, c_list_suite());

    return run_test_suite(suite, create_text_reporter());
}
// create_test_suite(); only needs to be done once.
// Add your own "suite" of tests by calling add_suite() and providing a pointer to your suite.
// In this case, c_list_suite() is created
</syntaxhighlight>

Now, in the '''''test_c_list.c''''' file, let's create the suite that we added to main() above.

<syntaxhighlight lang="c">
// test_c_list.c

#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>

// The file under test
#include "c_list.h"

TestSuite *c_list_suite()
{
    TestSuite *suite = create_test_suite();
    add_test(suite, add_element_to_end); // Sub-test, which we will add after.
    return suite;
}
</syntaxhighlight>

==== A Simple Test ====
Continuing on from our '''''test_c_list.c''''' file, let's perform one very basic test to verify the functionality of our C-code. We'll start by creating an Ensure() function. This is nothing more than a sub-routine test apart of the entire test suite we created. Typically, you will have an Ensure test function to test one specific functionality of your code. In this example, we only want to test the ability to add one element to the end of our linked list of c_list. The input argument to Ensure is simply a description of the test you're performing. Let's start this example by creating a pointer list.

<syntaxhighlight lang="c">
Ensure(add_element_to_end)
{
    c_list_ptr list = c_list_create();
    assert_that(c_list_node_count(list), is_equal_to(1));
}
</syntaxhighlight>

Compile the program by calling '''make''', which defaults to '''make all'''. To execute the program, simply perform '''./test_all.exe'''

<syntaxhighlight lang="c">
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 11ms.
code/test_c_list.c:15: Failure: c_list_suite -> add_element_to_end
        Expected [c_list_node_count(list)] to [equal] [1]
                actual value:                   [0]
                expected value:                 [1]

Completed "c_list_suite": 1 pass, 1 failure in 9ms.
Completed "main": 1 pass, 1 failure in 20ms.
</syntaxhighlight>

Oops. Looks like we got a unit test failure because we inserted a test assert that failed. The following line '''assert_that(c_list_node_count(list), is_equal_to(1));''' tests that the c_list ''list'' has a node list count of 1, but this failed because we didn't add any items into the ''c_list'', thus, this did not match our expectation of having a node list of 1. Let's go ahead and add that now and see if the test passes.

<syntaxhighlight lang="c">
Ensure(add_element_to_end)
{
    c_list_ptr list = c_list_create();
    c_list_insert_elm_end(list, (void*) 1);
    assert_that(c_list_node_count(list), is_equal_to(1));
}
</syntaxhighlight>

Compiling and running gives:

<syntaxhighlight lang="c">
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 14ms.
Completed "c_list_suite": 2 passes in 21ms.
Completed "main": 2 passes in 35ms.
</syntaxhighlight>

The output shows that by adding an element into the list returned a '''c_list_node_count''' of 1. We were able to test this by inserting a constraint on the assert on that specific unit test, which ultimately fails if the given inputs are not successful when ran. Here is a list of asserts and constraints you can use in your unit tests:

{| class="wikitable"
|-
! scope="col"| Assertion
! scope="col"| Description
|-
| assert_true(boolean)
| Passes if boolean evaluates true
|-
| assert_false(boolean)
| Fails if boolean evaluates true
|-
| assert_equal(first, second)
| Passes if 'first == second'
|-
| assert_not_equal(first, second)
| Passes if 'first != second'
|-
| assert_string_equal(char *, char *)
| Uses 'strcmp()' and passes if the strings are equal
|-
| assert_string_not_equal(char *, char *)
| Uses 'strcmp()' and fails if the strings are equal
|-
|}

Each assertion has a default message comparing the two values, but these can be substituted with your own failure messages, then you must use the '''*_with_message()''' counterparts...

{| class="wikitable"
|-
! scope="col"| Assertion
|-
| assert_true_with_message(boolean, message, …)
|-
| assert_false_with_message(boolean, message, …)
|-
| assert_equal_with_message(tried, expected, message, …)
|-
| assert_not_equal_with_message(tried, unexpected, message, …)
|-
| assert_string_equal_with_message(char *, char *, message, …)
|-
| assert_string_not_equal_with_message(char *, char *, message, …)
|-
|}

{| class="wikitable"
|-
! scope="col"| Constraint
! scope="col"| Passes if actual value/expression…
|-
| is_true
| evaluates to true
|-
| is_false
| evaluates to false
|-
| is_null
| equals null
|-
| is_non_null
| is a non null value
|-
| is_equal_to(value)
| '== value'
|-
| is_not_equal_to(value)
| '!= value'
|-
| is_greater_than(value)
| '> value'
|-
| is_less_than(value)
| '< value'
|-
| is_equal_to_contents_of(pointer, size)
| matches the data pointed to by pointer to a size of size bytes
|-
| is_not_equal_to_contents_of(pointer, size)
| does not match the data pointed to by pointer to a size of size bytes
|-
| is_equal_to_string(value)
| are equal when compared using strcmp()
|-
| is_not_equal_to_string(value)
| are not equal when compared using strcmp()
|-
| contains_string(value)
| contains value when evaluated using strstr()
|-
| does_not_contain_string(value)
| does not contain value when evaluated using strstr()
|-
| is_equal_to_double(value)
| are equal to value within the number of significant digits (which you can set with a call to significant_figures_for_assert_double_are(int figures))
|-
| is_not_equal_to_double(value)
| are not equal to value within the number of significant digits
|-
| is_less_than_double(value)
| < value withing the number of significant digits
|-
| is_greater_than_double(value)
| > value withing the number of significant digits
|-
|}

==== Another Unit Test (C++) ====
The first example showed a C function unit test. The following will serve as an example of how to test C++, or primarily ''classes''. Let's skip forward to the implementation of our simple foo.cpp, foo.hpp, turtle.hpp, and turtle.cpp files in the next sample unit test. We have a Foo class, which has a method called ''add'' that creates a local Turtle class object, calls ''draw'' and adds it to the input argument ''x''. The Turtle class method ''draw'' returns a constant value of 10 unconditionally.

<syntaxhighlight lang="c">
// foo.hpp & foo.cpp - Foo class that makes a call to a Turtle class
class Foo
{
	public:
    	int add(int x);
};

int Foo::add(int x)
{
	Turtle turtle;
	return x + turtle.turtle_draw();
}

// End foo.hpp & foo.cpp

// turtle.hpp & turtle.cpp - Random turtle class
class Turtle {
    public:
    	int turtle_draw(void);
};

int Turtle::turtle_draw(void)
{
	return 10;
}
// End turtle.hpp & turtle.cpp
</syntaxhighlight>

Next, let's skip forward to the '''''test_foo.cpp''''', where we will be testing the Foo module source file.

<syntaxhighlight lang="c">
#include <cgreen/cgreen.h>
#include <cgreen/mocks.h>
using namespace cgreen;

// The file under test
#include "foo.hpp"

// Other files we need to include
#include "turtle.hpp"

Ensure(testing_call_to_turtle)
{
    Foo foo;

    assert_that(foo.add(1), is_equal_to(21));
}

TestSuite *foo_suite()
{
    TestSuite *suite = create_test_suite();
    add_test(suite, testing_call_to_turtle);
    return suite;
}
</syntaxhighlight>

As you can see we have a unit test called ''testing_call_to_turtle'', where we will be testing the invocation of Foo.add(). The function ''add()'' makes an function call to another public method that is not located in the source file, Foo. Let's compile and execute the program now to analyze what happens under a normal scenario.

<syntaxhighlight lang="c">
/cygdrive/c/SJSU_Dev/projects/cgreen_test/example/foo.cpp:9: undefined reference to `Turtle::turtle_draw()'
</syntaxhighlight>

It looks like we got a compiler error. This means the unit test framework is working as expected. We need to set up the stub or mock for any external functions that this foo.cpp file calls. Adding in the function declaration with a return of the mock() function will allow us to do just so in this CGreen unit test framework.

<syntaxhighlight lang="c">
int Turtle::turtle_draw(void) { return mock(); }

Ensure(testing_call_to_turtle)
{
    Foo foo;
}
</syntaxhighlight>

Let's pause to take a break and go over what a mock is:
* Mock - a mock is a programmable object. In C objects are limited to function, and in C++ mocks are basically classes. The macro mock() compares the incoming parameters with any expected values and dispatches messages to the test suite if there is a mismatch. It also returns any values that have been preprogrammed in the test.

Let's go ahead and setup our expectations of any calls foo.draw will make. In this case, it calls turtle.draw so we have to set up 1 expect of turtle.draw for each foo.add performed.
<syntaxhighlight lang="c">
int Turtle::turtle_draw(void) { return mock(); }

Ensure(testing_call_to_turtle)
{
    Foo foo;

    // Expect the mock object, but return a value of 20. This overwrites the original implementation of turtle_draw
    expect(turtle_draw, will_return(20));

    // Test that when add(1) is called, the value 21 is returned based on our expect() we just made
    assert_that(foo.add(1), is_equal_to(21));
}
</syntaxhighlight>

This yields a successful output of all tests passing :

<syntaxhighlight lang="c">
$ ./test_all.exe
Running "main" (2 tests)...
Completed "foo_suite": 1 pass in 14ms.
Completed "c_list_suite": 2 passes in 11ms.
Completed "main": 2 passes in 25ms.
</syntaxhighlight>

Now, what happens if we change the return value of our expected mock object?

<syntaxhighlight lang="c">
int Turtle::turtle_draw(void) { return mock(); }

Ensure(testing_call_to_turtle)
{
    Foo foo;
    
    expect(turtle_draw, will_return(100));
    assert_that(foo.add(1), is_equal_to(21));
}
</syntaxhighlight>

<syntaxhighlight lang="c">
$ ./test_all.exe
Running "main" (2 tests)...
code/test_foo.cpp:25: Failure: foo_suite -> testing_call_to_turtle
        Expected [foo.add(1)] to [equal] [21]
                actual value:                   [101]
                expected value:                 [21]

Completed "foo_suite": 1 failure in 14ms.
Completed "c_list_suite": 1 pass, 1 failure in 20ms.
Completed "main": 1 pass, 1 failure in 34ms.
</syntaxhighlight>

As expected our test failed because the assertion result did not match because the return value was changed. 
This example demonstrates the fact that you can manipulate any external libraries or code that your source code depends on and you can test that it performs the way you expect it should.
Please, take a look inside '''''test_main.cpp''''' and you will find a #define where there are many examples of how you can test your code.

==== Sources and Extra References ====
* https://cgreen-devs.github.io
* http://cgreen.sourceforge.net/
* https://github.com/cgreen-devs/cgreen