Unit testing standards and best practices
From Post-Apocalyptic RPG wiki
This article covers how unit testing should be used in PARPG development.
What Is A Unit Test?
In general terms a unit test is a piece of code that "tests" the actual behavior of a unit of code against what you expect that code to do. There is no strict definition of a "unit" of code, rather it is a term used to describe any cohesive grouping of code that works together to achieve some common goal. In Python, this generally means that you will be writing unit tests for functions and methods, and (rarely) entire classes.
Unit tests are different from other tests like functional tests and integration tests in that:
- Unit tests work with the smallest possible unit of testable code, and doesn't really concern itself with whether or not a programmer will ever use that piece of code directly (its the indirect cases that count!);
- Ideally all unit tests run in isolation of each other, and thus produce results that are more easily debugged;
- Unit tests run quickly - if a unit test takes more than 10 seconds to run, then it is probably not a unit test;
Why Unit Test
Unit tests are essential tools for open-source projects because they provide immediate feedback to the code committer and to other developers in the project about whether committed code actually does what it says it does! In particular, thorough unit testing can:
- Catch bugs in your code before major commits, or at least alert developers that a potentially buggy commit was made so that it can be fixed quickly;
- Alert developers to regressions (reappearance of bugs that have already been fixed in the past), which could otherwise lead to debugging nightmares;
- Unit tests can act as detailed documentation for the code it tests;
Oh Nos, I Don't Wanna Write Unit Tests. They Take Too Long!
Unit testing should never be a chore - if it is, then you're doing it wrong (and hopefully we can guide you to write them the "right" way!). The main goal of unit testing is not just to test individual pieces of code and make sure they work right, but to make YOU think about your code and how it works. You may have spent countless hours planning and designing your code and think that it works exactly the way you want it to, but in reality that is rarely the case.
How To Write Unit Tests In PARPG
There are literally hundreds of ways to write unit tests, and that's probably why so many programmers don't really understand how to write effective unit tests. While we generally try to maintain a flexible workflow, we do have a set of guidelines on how to write unit tests to help simply things. Programmers should definitely adhere these guidelines whenever possible for consistency, but obviously they are not comprehensive.
Python unittest Module
PARPG uses the "PyUnit" or unittest module that is included in Python's standard library for its unit tests. A complete description of the unittest module is beyond the scope of this document, so checkout the Python 2.7 unittest documentation.
The basic premise of the unittest module is that you subclass the unittest.TestCase class that takes care of isolating the unit tests, setting up and tearing down test fixtures, providing methods of getting human-readable debugging information from the test results.
Initialization common to all test methods of a TestCase should be moved to the setUp() method of the TestCase subclass. Common finalization and cleanup should be moved to tearDown(). They will run before/after each test method.
Summary of Unit Test Workflow
- Write a module test suite;
- Write a class TestCase for each class in the module;
- Write a method TestCase for each method of each class;
- Write method tests for each method TestCase;
1. Write a Module Test Suite
A module test suite is a collection of unit tests of the functions and classes contained by a particular Python module. Each module test suite should reside in its own file named "test_<module_name>.py", and contain only unit tests of that module.
2. Write Class TestCases
For each class defined in the module being tested write a unittest.TestCase subclass named "Test<class_name>". This class TestCase will be used to define setUp, tearDown and helper methods common to all tests of the class' methods.
3. Subclass Method TestCases
For each method of the class being tested, write a subclass of the "Test<class_name>" TestCase defined above named "Test<method_name>". Each method TestCase can define its own setUp and tearDown methods or it can use the default methods defined by the class TestCase, allowing a great deal of flexibility in creating fixtures for the method tests.
4. Write Method Tests
Each method TestCase should then contain at least one method test named "test<DescriptiveTitle>", where "<DescriptiveTitle>" should roughly but concisely describe how the method is being tested and what the expected outcome should be (e.g. "testReturnsCorrectNumber"). Each method test must also define a one-line docstring that concisely describes the purpose of the test. This docstring is displayed in the unit test results, and should contain the name of the method being tested when appropriate.
Ideally there would be a method test for each possible path the code could take within the method. Each conditional statement you introduce into the method results in branching or an increase in the number of possible paths. Its not an exact science (though you can get pretty close by using code complexity metrics), but you should closely analyze your method and try to visualize what the different paths might be and the conditions that might lead to those paths being taken.
For example, if a user inputs a str instead of an int argument how do you expect your method to handle it? Should an exception be raised? Or should the error be silently handled by attempting to cast the str to an int? What happens if the cast also fails? These are all conditions that can and will happen, and each needs an appropriate method test. Any external state variables that are accessed or modified by the method can also lead to different code paths depending upon the value of that state variable.
- Several tests that verify the same kind of functionality and don't require object cleanup between them can be grouped in the same test method (e.g. testCoordinatesManipulation may have several groups of coordinates manipulations and assertions)
- Don't be afraid to use inheritance when designing method TestCases. If several of your method TestCases use the same setUp, tearDown or helper methods then put them in the class TestCase or subclass the class TestCase and override the setUp/tearDown methods to simplify things.