Your First Test

Test Directory

Let's create a Test Suite with a few assertions in it to see what TUnit can do. This test won't really be testing anything, it's just going to make some assertions and exit.

First, create a directory to store this test. This directory can go anywhere, but I'll put it in tinyos-2.x-contrib/tunit/tests/MyFirstTest for now:

 tinyos-2.x-contrib
 |-- tunit
 |   |-- tests
 |   |   |-- MyFirstTest

MyFirstTestC Configuration File

Create a configuration file, called MyFirstTestC.nc, and start writing it like you would any other TinyOS application. This will be the main entry point for your program.

 tinyos-2.x-contrib/tunit/tests/MyFirstTest/MyFirstTestC.nc
 
   0|  configuration MyFirstTestC {
   1|  }
   2|  
   3|  implementation {
   4|  
   5|    components new TestCaseC() as BasicAssertionTestC;
   6|  
   7|    components MyFirstTestP;
   8|  
   9|    MyFirstTestP.BasicAssertionTest -> BasicAssertionTestC;
  10|  
  11|  }

On line 5, we create a new instance of TestCaseC. This is saying "Add a new test to our suite". Notice we immediately rename all instances of TestCaseC's as SomeReallyLongAndDescriptiveTestNameC. In this case, we called it BasicAssertionTestC.

 TUnit Tip
 It's important to rename your TestCaseC instances with descriptive names. 
 When that test fails, TUnit will magically tell you the name of the offending 
 test so you can go look it up.

If you're worried about what the TestCaseC component actually refers to, it comes from the TUnit embedded libraries. The library is found in tinyos-2.x-contrib/tunit/tos. TUnit will take care of referencing this entire library for you at compile time so you don't have to think about it.

The signature of the TestCaseC component looks like this:

 generic configuration TestCaseC() {
   provides {
     interface TestCase;
     interface TestControl as SetUpOneTime @atmostonce();
     interface TestControl as SetUp @atmostonce();
     interface TestControl as TearDown @atmostonce();
     interface TestControl as TearDownOneTime @atmostonce();
   }
 }

The interface we're interested in to run our basic assertion test is the TestCase interface. The TestCase interface is defined in tinyos-2.x-contrib/tunit/tos/interfaces/TestCase.nc:

 interface TestCase {
 
   event void run();
   
   async command void done();
 
 }

Pretty simple interface, right? Your TestCase signals you to run(), and you must call back when your test is done(). With that in mind, we have a module to build...

MyFirstTestP Module File

As the configuration file above dictated, we need to have a module called MyFirstTestP that uses the interface TestCase as BasicAssertionTest. Let's create the MyFirstTestP module now:

 tinyos-2.x-contrib/tunit/tests/MyFirstTest/MyFirstTestP.nc
 
   0|  #include "TestCase.h"
   1|  
   2|  module MyFirstTestP {
   3|    uses {
   4|      interface TestCase as BasicAssertionTest;
   5|    }
   6|  }
   7|  
   8|  implementation {
   9|  
  10|    event void BasicAssertionTest.run() {
  11|      assertSuccess();
  12|      call BasicAssertionTest.done();
  13|    }
  14|  
  15|  }

There are a couple things to point out here.

Line 0 is the all-important TestCase.h import. This header file allows TUnit to make assertions. The TestCase.h file is mixed in with the TUnit embedded library. Don't think about it, just import it and refer back to the list of assertions when necessary.

Line 10 is the entry point for our first test. TUnit will tell each test when to run(), and your responsibility is to tell TUnit when that test is done(). TinyOS is split-phase, so there's no way for TUnit to know when your test is actually done without you explicitly saying so.

Inside the test, line 11 makes an assertion. In this case, we assertSuccess(), which is the equivalent of saying "If the test made it this far in the code, the test was a success."

Each Test Case must make at least one assertion. If a test doesn't make any assertions, TUnit will give you a warning. It's possible your test failed to execute correctly and test what it was supposed to test.

Line 12 is very important. I cannot stress this enough: when your test is done running, you must tell TUnit that the test is done(). This deserves a huge tip comment on it's own:

 TUnit Tip
 TUnit's tells your tests when to run().
 After receiving the signal to run() and your test is completes,
 you MUST call back to TUnit telling it your test is done()!

If we didn't call BasicAssertionTest.done() on line 12, TUnit would have timed out after a default of 1 minute (expandable using suite.properties rules, described later). This timeout would normally occur if the test locked up your microcontroller, allowing the TUnit framework on the PC side to continue safely executing other tests.

 TUnit Tip
 If TUnit times out for apparently no reason,
 you probably forgot issue a done() command somewhere.

Makefile

Just like every other TinyOS application, this one needs a Makefile so it can be compiled:

 tinyos-2.x-contrib/tunit/tests/MyFirstTest/Makefile
 
   0|  COMPONENT=MyFirstTestC
   1|  include $(MAKERULES)

Notice we don't try to include anything in the Makefile except the name of the entry test configuration, and the makerules reference. Don't try to include compile flags or anything else in the Makefile except these two lines. The suite.properties file is the place to put compiler options, and is discussed later.

Also, TUnit is responsible for pulling in the TUnit embedded libraries, so don't worry about telling the compiler where they are located.

At this point, your directory structure should look like this:

 tinyos-2.x-contrib
 |-- tunit
 |   |-- tests
 |   |   |-- MyFirstTest
 |   |   |   |-- Makefile
 |   |   |   |-- MyFirstTestC.nc
 |   |   |   |-- MyFirstTestP.nc

Run TUnit

Go to your command line and drill down to the MyFirstTest directory. Make sure you have your motes plugged in and the tunit.xml file configured correctly (see Setting up TUnit), and run the TUnit Java application (which I have aliased "tunit"). Here is a truncated version of what I see when I run the test:

 $ cd /opt/tinyos-2.x-contrib/tunit/tests/MyFirstTest
 $ tunit
 0 [main] INFO com.rincon.tunit.TUnit  - Base package directory located: C:\
 15 [main] INFO com.rincon.tunit.TUnit  - Found a TOSCONTRIB environment variable
 
 140 [main] INFO com.rincon.tunit.TUnit  - STARTING TEST RUN TELOSB
 
 Now we're actually running our first Test Run, check if the motes exist.
 If the motes in this test run were found, find a test to compile.
 When a test is found in your current directory or a sub-directory, compile it
 and install it to the node attached to your computer.
 
     compiling MyFirstTestC to a telosb binary
 ncc -o build/telosb/main.exe -Os -O -mdisable-hwmul -Wall -Wshadow -DDEF_TOS_AM_
 GROUP=0x7d -Wnesc-all -target=telosb -fnesc-cfile=build/telosb/app.c -board= -Ic
 :/tinyos/cygwin/opt_svn/tinyos-2.x-contrib/tunit/tos/lib/tunit -Ic:/tinyos/cygwi
 n/opt_svn/tinyos-2.x-contrib/tunit/tos/lib/tunitstats -Ic:/tinyos/cygwin/opt_svn
 /tinyos-2.x-contrib/tunit/tos/system -Ic:/tinyos/cygwin/opt_svn/tinyos-2.x-contr
 ib/tunit/tos/interfaces -Ic:/tinyos/cygwin/opt_svn/tinyos-2.x-contrib/tunit/tos/
 lib/directserial -Ic:/tinyos/cygwin/opt_svn/tinyos-2.x-contrib/tunit/tos/lib/fif
 oqueue -DTUNIT_TOTAL_NODES=1  MyFirstTestC.nc -lm
     compiled MyFirstTestC to build/telosb/main.exe
             7200 bytes in ROM
              471 bytes in RAM
 
 After installing the application to the node, make sure we can communicate
 with the mote over the serial forwarder by sending a ping.
 If a pong is received (hidden), kick off the test.
 
 41085 [Thread-19] INFO com.rincon.tunit.run.ResultCollector  - Test 0 (MyFirstTe
 stC.BasicAssertionTestC) passed
 
 The test is complete, clean everything up and find the next test to run.
 In this case, there are no more tests to run, so we exit and print all the results
 at the end.
 
 
 T-Unit Results
 -----------------------------------------------
 Total runtime: 44.974 [s]
 Total tests recorded: 2
 Total errors: 0
 Total failures: 0
 

Notice that there was only a single Test Case making a single assertion, but 2 tests were recorded. This is because the compilation process is a test itself. Because we compiled successfully, that test passed and increases our total tests recorded.

Assertion Playground

Let's try out some different assertions by modifying our MyFirstTestP.nc file. Remember to refer back to TUnit Assertions to see what types of assertions can be made.

assertFail()

We'll simply change our assertSuccess() into assertFail(<fail msg>):

 /opt/tinyos-2.x-contrib/tunit/tests/MyFirstTest/MyFirstTestP.nc
   
   0|  #include "TestCase.h"
   1|  
   2|  module MyFirstTestP {
   3|    uses {
   4|      interface TestCase as BasicAssertionTest;
   5|    }
   6|  }
   7|  
   8|  implementation {
   9|  
  10|    event void BasicAssertionTest.run() {
  11|      assertFail("Failure!");
  12|      call BasicAssertionTest.done();
  13|    }
  14|  
  15|  }

Every test failure requires you to give a textual reason why the test failed. In this case, we'll print out the string "Failure!" because our test is now failing at line 11. The assertFail(<fail msg>) is the equivalent of saying "If we reach this line of code, our test failed."

 TUnit Tip
 These microcontrollers don't have a lot of memory.
 Although longer messages are ok (requiring several packets to be delivered to
 the computer to report the whole message), keep your failure descriptions short.
 Succinct messages are less likely to be truncated.

Now let's run TUnit...

 $ tunit
 
 ...
 
 T-Unit Results
 -----------------------------------------------
 Total runtime: 41.922 [s]
 Total tests recorded: 2
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): Failure!

assertEquals(), assertNotEquals(), assertTrue(), assertFalse()

We can verify two values are the same. Notice in this test, we'll make multiple assertions. TUnit, by default, has a queue for 5 assertion messages to get across to the computer in a single Test Case. This queue is flushed completely between tests or between task posts. The suite.properties file allows you to increase the number of assertions each Test Case is allowed to make, which we'll show in a minute.

 /opt/tinyos-2.x-contrib/tunit/tests/MyFirstTest/MyFirstTestP.nc
   
   0|  #include "TestCase.h"
   1|  
   2|  module MyFirstTestP {
   3|    uses {
   4|      interface TestCase as BasicAssertionTest;
   5|    }
   6|  }
   7|    
   8|  implementation {
   9|  
  10|    event void BasicAssertionTest.run() {
  11|      assertEquals("uint16_t isn't 2 bytes", 2, sizeof(uint32_t));
  12|      assertTrue("False!", TRUE);
  13|      assertFalse("True!", FALSE);
  14|      assertNotEquals("Values rounded off!", (float) 4.59, (int) 4.59);
  15|      call BasicAssertionTest.done();
  16|    }
  17|  
  18|  }

Notice that on line 11 and 14, the value you expect is the first argument, and the result you actually obtained is the second. This format applies to every other unit testing framework you'll use. Running TUnit, all our tests pass.

 T-Unit Results
 -----------------------------------------------
 Total runtime: 42.441 [s]
 Total tests recorded: 5
 Total errors: 0
 Total failures: 0

Failure in assertEquals()

Changing line 11 to assertEquals("uint16_t isn't 2 bytes", 2, sizeof(uint32_t));, we see some failure output that helps us locate the problem:

 T-Unit Results
 -----------------------------------------------
 Total runtime: 43.127 [s]
 Total tests recorded: 5
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): uint16_t isn't 2 bytes; Expecte
 d [2] but got [4] (unsigned 32-bit form)

The comment about the unsigned 32-bit form is there to help you remember that any numbers displayed are all converted to uint32_t's. If we were to change line 11 to assertEquals("Wrong number", -2, -3);, then we'd see some ridiculous output due to the interpretation of the sign:

 T-Unit Results
 -----------------------------------------------
 Total runtime: 42.361 [s]
 Total tests recorded: 5
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): Wrong number; Expected [4294967
 294] but got [4294967293] (unsigned 32-bit form)

If you were to manually convert those numbers back to a signed number, you'd see they're still -2 and -3.

Failure in assertTrue()

If we were to change line 12 to assertTrue("False!", FALSE);, then we'd get...

 T-Unit Results
 -----------------------------------------------
 Total runtime: 42.066 [s]
 Total tests recorded: 5
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): False!

Failure in assertNotEquals()

If we changed line 14 to assertNotEquals("Values rounded off!", (float) 4.0, (int) 4.59);, then the values both round off to 4 and become equal, which fails our assertion:

 T-Unit Results
 -----------------------------------------------
 Total runtime: 42.018 [s]
 Total tests recorded: 5
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): Values rounded off!; Shouldn't
 have gotten [4] (unsigned 32-bit form)

Notice, though, that the types of arguments being compared doesn't matter. Here we're comparing a float and an int, and it's possible to compare any other type combination. For arrays or structures, we have to use assertCompares().

assertResultIsAbove(), assertResultIsBelow()

The assertResultIsAbove() and assertResultIsBelow() are only > and < tests; they are not >= or <=. These are typically used to set a threshold of minimum performance before the test is considered a failure.

Too many assertions!

Again, by default there is a queue of 5 assertion messages that can get through to the computer. Each time an assertion is made, one of the messages is eaten up. If an assertion fails and requires a multi-packet failure message, then multiple packets from the queue get eaten up. Until a task is posted or the test is complete, messages will not be sent from the queue.

Let's see what happens when our test makes too many assertions...

 /opt/tinyos-2.x-contrib/tunit/tests/MyFirstTest/MyFirstTestP.nc
 
   0|  #include "TestCase.h"
   1|  
   2|  module MyFirstTestP {
   3|    uses {
   4|      interface TestCase as BasicAssertionTest;
   5|    }
   6|  }
   7|  
   8|  implementation {
   9|  
  10|    event void BasicAssertionTest.run() {
  11|      assertFail("This is a really long failure message that spans multiple messages and eats up the message queue.");
  12|      assertFail("Short msg");
  13|      assertSuccess();
  14|      call BasicAssertionTest.done();
  15|    }
  16|  
  17|  }

And the result is...

 T-Unit Results
 -----------------------------------------------
 Total runtime: 42.285 [s]
 Total tests recorded: 2
 Total errors: 0
 Total failures: 1
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): This is a really long failure m
 essage that spans multiple messages and eats up t

Notice only 2 tests were recorded. Of those, one of them was the test to see if we could compile. Our test made three assertions, but only the first one got through. Rude, but repairable. We'll need to access the suite.properties file to increase the number of elements in the TUnit queue.

suite.properties File

Create (or copy from another directory) a file called "suite.properties". This file contains all the rules TUnit should use when running the test in the current directory or sub-directories. Read the suite.properties file documentation for all the rules you can include.

 /opt/tinyos-2.x-contrib/tunit/tests/MyFirstTest/suite.properties
 
 0|  @assertions 20
 1|  

That's it!

 TUnit Tip
 Each extra assertion in TUnit's embedded side queue takes up memory.
 Keep the assertion queue low, and your test won't crash on smaller platforms.

Now let's run the test above with the suite.properties addition...

 T-Unit Results
 -----------------------------------------------
 Total runtime: 41.92 [s]
 Total tests recorded: 4
 Total errors: 0
 Total failures: 2
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): This is a really long failure m
 essage that spans multiple messages and eats up the message queue.
 
 MyFirstTestC.BasicAssertionTestC (EmbeddedTest): Short msg

The total of 4 tests came from the 1 compile test, plus the 3 assertions made on the embedded side. We also see the full long message printed out and the short failure message afterward.