
January 13, 2004
Introduction
In 1996, the European Space Agency watched in horror as the maiden flight of
the Ariane 5 rocket automatically self-destructed 39 seconds into its flight.
In 2001, NASA (in conjunction with the European Space Agency) was forced to
admit that their $125 million Mars Climate Orbiter had instead become Mars Surface
Debris. The link (other than the ESCA) is that both of these catastrophes were
caused by software failures, and both were projects developed by some of the
most rigorous programming method practitioners known to the industry.
Most software development projects follow a standard pattern for testing:
- Programmers develop code, which is checked into a common respository.
- At regularly scheduled intervals, and during major releases, the Quality
Assurance department builds the application and tests it (sometimes manually,
sometimes using automated scripts).
- Defects and issues are reported back to the programmer, describing the offending
behavior.
- The programmer spends 4 hours tracking down exactly where the bug occurs
in the code, only to discover a simple error which takes 15 seconds to fix.
- The code is checked back in, and the cycle repeats.
Having a dedicated QA department and regularly scheduled testing is vital to
a project’s success. It is also the least efficient way to test
an application. It requires the cessation of activity from the programmer, dedicated
human testers examining the entire application, and a lengthy interval between
the writing of buggy code and discovering the bug. Perhaps worst of all, this
kind of testing can really only identify gross failures; the details are left
to be ferreted out by the programmer.
Let’s compare software development to another kind of authorship: writing
a novel. An author, upon finishing a draft of a novel, will send it to the editor
for review. The editor (and probably a team of reviewers) read the novel to
make sure it makes sense, has no major flaws, and is fun to read. This is the
acceptance testing phase for the novel; it takes the reviewers a long
time to read the draft, and they are focused on large issues of plot and structure.
It is possible that during the writing of the novel, the author has been shipping
individual chapters to the editor for review. In this case, there is a shorter
interval between the writing and the reviewing. It takes less time to read a
single chapter than a whole novel, and the issues reported are much more specific,
although still focused on issues of structure and plot. This is a kind of module
testing.
Imagine, though, relying on these review phases for catching problems like
spelling and grammar mistakes. For hundreds of years, that is exactly what authors
and editors did. The review process handled both the larger issues of plot and
structure and the detailed issues like spelling and grammar. As such, reviews
took much longer, the feedback was quite lengthy, and many small errors went
unnoticed until after the novel was shipped. What authors needed was something
that they could employ themselves to give them immediate feedback on the individual
units (words) of the book as they wrote; something that was automatic, that
they could trust to catch the majority of mistakes. Enter the spellchecker.
Writing a novel using a modern word processor, authors get immediate feedback
on their spelling and grammar. Hundreds of minor mistakes are corrected the
moment they are made. They won’t catch everything, but they don’t
have to. The resulting manuscript is so much cleaner by the time it gets to
the reviewers that they can focus their energies on the things that can’t
be caught by a spellchecker. The author is freed from the psychological burden
of seeing page after page of errors to fix. The resulting product is that much
stronger.
The spellchecker is for the author a kind of unit testing. Unit testing
is the act of writing test code to verify your own production code. Unit testing
is done by the programmer, and the immediate benefit is to the programmer. Each
individual unit of functionality (method) is tested, in isolation where possible,
to make sure that the individual building blocks of your application are solid.
By limiting the amount of functionality being tested, and controlling the environment
in which it is tested, you can verify the code itself while minimizing the unpredictable
effects of context. The cost to the team is orders of magnitude less than with
end-of-cycle testing.
Perhaps more importantly, unit testing offers the shortest feedback cycle between
writing code and finding out if you wrote it incorrectly. Bugs are always easiest
to fix when the code is fresh. Waiting until end-of-cycle testing gives you
an opportunity to forget why and how you wrote a specific unit of functionality.
Unit testing does not necessarily require any special technologies. However
one practices the craft, unit tests must merely be:
- Automatic (they check their own results)
- Repeatable (able to be run again and again, by multiple people)
- Available (they should accompany the code they test, so anybody who has
that code can run them)
For our novelist, using a word processor with a spellchecker, they get automatic
feedback (usually as they type), which they can repeat at will (you can always
invoke the spellchecker manually) and which is available to the reviewers as
well (all they need is a copy of the word processing software). A spellchecker
can’t catch everything (homonyms are a classic case), but they catch enough.
For a programmer, unit testing is a little more complex because you have to
write the tests yourself. However, as long as you have a reasonable mechanism
for achieving those three goals, you can perform unit testing any way you desire.
A good, standardized framework can provide a lot of features for very little
effort. NUnit, an open source unit testing framework for .NET, is available
at www.nunit.org, and provides
a group of test running applications and tools for reporting the results.
Unit Testing and the Development Team
Writing unit tests as you write production code gives you instant feedback
insight into your successes and failures. In a distributed team environment,
this alone is invaluable. If each developer is confirming the stability of their
code before checking it in to version control, then all developers can know
that a certain level of quality is assured. Egregious careless errors are being
eliminated before any code is integrated into the full project.
In the case of distributed team development, though, unit testing code provides
another, more powerful benefit. Since the testing mechanism is simply more code,
it gets checked into version control as well. Any build process that collects
the code from the various developers can also run the unit tests for their components
and modules. The tests, which were once unit tests, are now integration and
regression tests as well. If tests that passed on a developer's machine in isolation
now fail after a full build on the integration server, then there is an unplanned
side-effect or interaction between modules from two different developers. If
tests that passed on previous integration builds now fail, then there is an
unplanned side-effect from new code that is rippling through the existing codebase.
Since most unit testing is done using a standardized framework like NUnit,
the tests can be run automatically by an integration tool (such as NAnt or make)
and results sent to the team members. What began as a way of verifying your
own code before adding it to the project at large has become a way of verifying
the whole project at each build.
Using NUnit
NUnit, and the xUnit family of testing frameworks, all provide roughly the
same functionality. They allow you to write test code in the same language as
your production code, and they each provide a collection of test runners, applications
which automate the process of running the tests and reporting on the results.
NUnit was originally a .NET-based clone of JUnit, a very popular framework for
Java. However, since version 2.0, NUnit has been completely reworked to take
advantage of the unique features of the .NET platform, namely extensible metadata
(attributes).
Creating unit tests with NUnit requires making a reference to the central NUnit
library, nunit.framework.dll. Then, you write code which you decorate with a
series of attributes provided by the framework. These attributes are how you
designate tests, organize them into groups called suites, and control their
lifecycle. The test runners provided by NUnit then look for those attributes
when conducting a pass through your code.
Running Tests
NUnit provides two test runners for you: a winforms application called nunit-gui.exe,
and a console-based application called nunit-console.exe. Both provide visual
feedback about the results of the tests, both allow you to persist the results
as xml, and both allow you to pass in assemblies to test as command-line arguments.
The big difference is whether or not you get a big hierarchical tree-view of
the results (nunit-gui) or a scrolling list of textual results (nunit-console).
The GUI application is better for verifying your own code in your development
environment while you work on it. The console version is more useful for automated
testing environments, like a build tool (NAnt or other) on your build and integration
server. Neither is particularly well integrated with Visual Studio .NET. Certainly,
you can configure either as an external tool with key mappings and menu items
for launching them, but it is difficult to have the tests run automatically
with each build, and the reporting is handled through an external application.
What you really need is a built-in tool, part of VS.NET, which runs tests whenever
your project is built and gives you instant feedback.
Integrating with Visual Studio (VSNunit)
VSNUnit is an open-source add-in for Visual Studio .NET developed by the author
that provides instant, automated testing and feedback. It is available at http://www.relevancellc.com/vsnunit.htm.
Once you install VSNunit, you now have a dockable toolwindow for displaying
the output of your unit tests. VSNunit will automatically recognize which version
of NUnit your project is using (though you can always override by choosing a
version manually). You can configure VSNunit to either run manually (by pressing
the Start Tests button) or to run automatically after each successful build
of the project.

Figure 1. VSNunit integration with Visual Studio. (click to see full size image)
When the tests are run, the results are shown in the classic NUnit tree style.
Tests are nested within test suites, which in turn are nested inside test fixtures.
A successful test has a green light next to it; failed tests get a red light.
A failed test will show two child nodes: the failure message, and the stack
trace of the failure. Double clicking on a failed test will highlight the line
of code in the editor window where the failure occurs (opening the code file
first if necessary).
You can cycle through the failed tests by pressing the Next Failure
and Previous Failure buttons. This allows you to quickly move to all the failed
tests, without having to scroll around in the result tree looking for them.
Finally, you can persist the results to an XML file using the Save button.
It can often be useful to save individual results either for historical comparison,
or for sharing with other developers.
Assertions
Unit tests are really just a collection of true/false propositions. They are
blocks of code that exercise other blocks of code, and verify that certain conditions
are (or are not) met. The true/false propositions are called assertions.
NUnit provides a static object called Assert that provides a series of static
methods, all of which evaluate a Boolean condition. Whenever the Boolean condition
evaluates to true, the assertion succeeds. When it evaluates to false, the assertion,
and thus the test, fails. Any single failed assert within a given test causes
the entire test to fail.
The most prevalent form of assertion is Assert.IsTrue,
which takes a Boolean and succeeds if that Boolean is true. Its logical opposite
is Assert.IsFalse, which
succeeds if the Boolean parameter evaluates to false. Assert.IsNull
and Assert.IsNotNull
check to see if the inbound parameter has been initialized. Assert.AreEqual
checks if two inbound parameters have equivalent values; Assert.AreSame
checks if two inbound parameters are references to the same object. Assert.Fail
is simply a guaranteed failure. Each Assert method has an optional string parameter
for describing what the assertion was testing, making analysis of the reported
results much easier.
An Example: CPasswordManager
In order to demonstrate the power of unit testing, we'll need a concrete example
to walk through. For this example, we will use a class for managing user passwords
in a custom user database. The requirements are:
1. Implement the CPasswordManager class, which: a. Exposes a MeetsPasswordPolicy method, which: i. Takes a string parameter, “password” ii. Returns a Boolean, representing whether the password passes the policy test iii. Compares the password to a stored policy to determine the correct response b. Exposes a public PasswordPolicyException class, which: i. Derives from ApplicationException ii. Has no custom constructor c. Exposes a HashPassword method, which: i. Takes a string parameter, “password” ii. Returns a string, the hashed version of the password iii. Throws a new PasswordPolicyException instance if the password fails the policy test iv. Computes the hash of a valid password input using the SHA1 CryptoServiceProvider
Tests
So, what is a test? A test is merely a parameter-less method that has been
decorated with the [Test]
attribute (in the NUnit.Framework
namespace) and contains zero, one or more assertions. If a test has no assertion
statements, that means it is relying on built-in exceptions to determine success
or failure. Unhandled exceptions are treated as failed assertions.
Let’s look at testing the MeetsPasswordPolicy method:
public bool MeetsPasswordPolicy(string password)
{
bool meetsPolicy = false;
//NOTE: this policy is too simple; should probably be a
// regular expression, checking for length and mandatory
// character types
if(password.Length >= 8)
meetsPolicy = true;
return meetsPolicy;
}
The method itself takes a password as a string and returns true or false depending
on the application of the password policy. For completeness, the test of this
method should include passing in passwords that are known to pass the policy
check, known to fail the policy check, and represent odd boundary conditions.
[Test]
public void TestPolicy()
{
private CPasswordManager hash = new CPasswordManager();
Assert.IsTrue(hash.MeetsPasswordPolicy("goodpass"),
"Good password failed check");
Assert.IsFalse(hash.MeetsPasswordPolicy("badpass"),
"Bad password passed check");
}
This test determines that a password (“goodpass”) of at least 8
characters passes the check, and one of less (“badpass”) fails.
However, our test is incomplete. What about the odd boundary conditions? In
this case, we should test at least for passing the empty string (“”)
and a null string.
[Test]
public void TestPolicy()
{
private CPasswordManager hash = new CPasswordManager();
Assert.IsTrue(hash.MeetsPasswordPolicy("goodpass"),
"Good password failed check");
Assert.IsFalse(hash.MeetsPasswordPolicy("badpass"),
"Bad password passed check");
Assert.IsFalse(hash.MeetsPasswordPolicy(""),
"Empty password passed check");
Assert.IsFalse(hash.MeetsPasswordPolicy(null),
"Null password passed check");
}
Running this set of tests will expose a flaw in the code; the null password
causes an exception of type
NullReferenceException, which is reported as a failure of the TestPolicy
test.. The MeetsPasswordPolicy
method simply checks the Length attribute of the password parameter without
first ensuring that it is not null. To solve the problem, change
MeetsPasswordPolicy to look like this:
public bool MeetsPasswordPolicy(string password)
{
if(password == null)
return false;
bool meetsPolicy = false;
//NOTE: this policy is too simple; should probably be a
// regular expression, checking for length and mandatory
// character types
if(password.Length >= 8)
meetsPolicy = true;
return meetsPolicy;
}
Unit tests are most valuable when they expose flaws and inconsistency in the
code base; in order to do that, you should contemplate the common conditions
that could exist during the execution of the method under test.
Handling Exceptions
NUnit has a simple algorithm for determining whether a test passes or fails:
if all the assertions pass, the test passes. If any assertion fails, the test
fails. But what happens if a line of code happens to throw an exception? The
good news is that NUnit recognizes unhandled exceptions for what they are: an
indicator that the code has failed in some way. As such, it reports unhandled
exceptions as failures of the test itself.
However, it is sometimes true that we want code to throw an exception; when
our design calls for the propagation of that exception to the calling code.
Take our requirements for CPasswordManager.HashPassword,
for example. When an invalid value is passed in for the password, our code should
throw a PasswordPolicyException.
Since this is expected behavior, we don’t want the test to fail (like
with the NullReferenceException from the previous section).
In order to allow our test to pass, to treat the exception as an expected result,
we need a new attribute for our test method. [ExpectedException]
takes a single parameter, a Type representing an exception that is supposed
to be thrown. Here is the full definition for TestHashPassword:
[Test, ExpectedException(typeof(PasswordPolicyException))]
public void TestHashPassword()
{
private CPasswordManager hash = new CPasswordManager();
string cache = hash.HashPassword("MyPassword");
Assert.IsTrue(cache.Length > 0,
"Hash returned zero-lengh result");
Assert.IsTrue(cache != "MyPassword",
"Hash returned input string");
Assert.AreEqual(cache,
hash.HashPassword("MyPassword"),
"Second hash of MyPassword different from first");
Assert.IsTrue(cache != hash.HashPassword("mypassword"),
"Hash of mypassword same as of MyPassword");
cache = hash.HashPassword("badpass");
Assert.Fail("Should have trigged CPasswordPolicyException");
}
Here, we are testing for a valid hash result given a good input password, and
verifying that the result is repeatable, and different for different inputs.
Finally, we verify that a password which does not meet the password policy generates
a PasswordPolicyException.
Notice the last two lines of the method:
cache = hash.HashPassword("badpass");
Assert.Fail("Should have trigged CPasswordPolicyException");
The first line attempts to hash a password of only 7 characters. It should generate
a PasswordPolicyException.
Due to the way .NET handles exceptions, control should immediately pass out of
the executing context to the containing component, in this case, the NUnit test
runner. NUnit then examines the exception to see if it is of the expected type,
and if so, passes the test. Otherwise, the test fails and the exception is reported.
Since control passed to the containing component, the second line with Assert.Fail
will never be called. We leave this line in our test, however, to show us if the
HashPassword method ever
stops throwing the exception given a 7 character password. If that Assert.Fail
ever triggers, the test itself fails and we will be notified.
TestFixtures
A test fixture is a class that contains a series of test methods. The class
is decorated with the [TestFixture]
attribute. Test fixtures are a way of organizing logically grouped tests. This
grouping is important for two main reasons: one, it creates a subset of tests
that can be run as a group, separate from other groups of tests, and two, it
provides a level of granularity for controlling the lifecycle and context of
a test.
Each individual test method, it turns out, must be able to exist on its own.
Tests should not be affected by side effects from other tests, nor should they
cause side effects to occur. In other words, tests should live in a stateless
environment. When we adhere to this rule, we ensure that we are only testing
one functional unit at a time, that we do not propagate error conditions between
tests, and that tests can be run independently, in any order.
Our test fixture looks like this:
[TestFixture]
public class CTestPasswordManager
{
//...TestHashPassword(), TestPolicy()
}
The Test Lifecycle (SetUp and TearDown)
If you examine the tests we have written, you will notice some repetitive code.
Each and every one has to create an instance of a CPasswordManager
in order to perform its tests, and many of the individual tests re-use the same
literal values (“MyPassword”, “badpass”). Repetitive
code is a nasty thing; it leads directly to bugs. A simple solution would be
to create a set of class-scoped variables for use by all the tests:
[TestFixture]
public class CTestPasswordManager
{
private CPasswordManager hash = new CPasswordManager();
private const string mypassword = "MyPassword";
private const string mypassword2 = "mypassword";
private const string goodpassword = "goodpass";
private const string badpassword = "badpass";
[Test]
public void TestPolicy()
{
Assert.IsTrue(hash.MeetsPasswordPolicy(goodpassword),
"Good password failed check");
//etc.
}
}
This arrangement seems to solve the problem. Now, each of the test methods can
take advantage of these class-scoped variables, and ensure that each test is using
the right values. For the string values, this is appropriate, as long as you remember
to make them constants. Remember, one of the primary properties of a unit test
is its isolation from all other unit tests. This means that changes to some global
state that occur inside one test should never affect the execution of another.
Constant variables enforce this, since their values are immutable.
For reference types, however, a better solution is to use [SetUp]
and [TearDown] methods.
Each is a parameter-less method decorated with the appropriate attribute. NUnit
guarantees that the SetUp
method in a class will be run just before each test in the TestFixture,
and the TearDown method
will be run just after each test. This provides you the opportunity to create
whatever pristine state you want for each test to operate in.
[TestFixture]
public class TestPasswordManager
{
private CPasswordManager hash;
//...Class-scoped, constant values
[SetUp]
public void Setup()
{
hash = new CPasswordManager();
}
[TearDown]
public void Teardown()
{
hash.Dispose();
hash = null;
}
//Tests, etc.
}
Handling Unfinished Tests
In larger systems, with more complex development teams, it is possible that
unit tests can be written by different programmers than the production code
they test. This can mean that extensive libraries of unit tests exist long before
any production code for them to exercise. Depending on how strict your internal
rules are about what happens when tests fail, you will definitely want a way
to explain why some tests don't work.
NUnit allows you to have tests or test fixtures that are part of the library,
but not run with the rest of the tests. Simply decorate the method or class
with the [Ignore] attribute,
passing in a string message explaining why the test is being ignored.
[Test, Ignore("Have not implemented GeneratePassword yet.")]
Public void TestGeneratePassword
{
string newPassword = hash.GeneratePassword();
Assert.IsNotNull(newPassword, "No password returned.");
Assert.IsTrue(newPassword.Length > 0,
"Password returned was zero-length.");
Assert.IsTrue(hash.MeetsPasswordPolicy(newPassword),
"Password does not meet policy.");
}
When you run the tests, ignored tests are reported separately from passes and
fails, and the reasons are displayed in the report. Now, you aren't merely getting
validation of code you write, but progress reports on the total codebase. The
reports can tell you what is working, what is broken, and what hasn't even been
addressed yet.
Testing in Enterprise Development
Testing Database Code
A good many applications written today are based on interacting with a database.
Code that interacts with the database can benefit from unit testing just like
any other block of code. Testing against a database has an inherent problem,
though: unit tests should operate in a stateless manner, and a database’s
raison d’être is to maintain stateful information. Programmers
must make sure that they test functionality in isolation as much as possible.
To do so, here are three simple rules:
- Never run unit tests against a production database. Always use a database
assigned specifically for testing, whose data you can modify at will. Create
the database using the schema from the production database, but leave its
data empty.
- Create generic stored procedures that initialize the database for a unit
test. Using the SetUp and TearDown methods of your fixture, you can initialize
the database to a known state before each test, then remove all data afterward.
[TestFixture]
public class TestDatabase
{
[SetUp]
public void Setup()
{
SqlConnection conn = new
SqlConnection("SOME_CONN_STRING");
SqlCommand comm = conn.CreateCommand();
comm.CommandText = "spInitializeAllTables";
comm.ExecuteNonQuery();
conn.Dispose();
}
[TearDown]
public void Teardown()
{
SqlConnection conn = new
SqlConnection("SOME_CONN_STRING");
SqlCommand comm = conn.CreateCommand();
comm.CommandText = "spClearAllTables";
comm.ExecuteNonQuery();
conn.Dispose();
}
}
- Create more specific stored procedures that add additional data for
specific tests. These stored procedures should assume that the generic procedure
has already been run. Call them at the beginning of the specific method that
requires them. As always, do not repeat yourself. If there are ways to share
these procedures among multiple methods, do so.
[TestFixture]
public class TestDatabase
{
//Setup and teardown
//...
[Test]
public void TestWhenDuplicateUserNames()
{
SqlConnection conn = new
SqlConnection("SOME_CONN_STRING");
SqlCommand comm = conn.CreateCommand();
comm.CommandText = "spAddDuplicateUserNames";
comm.ExecuteNonQuery();
conn.Dispose();
//Perform actual tests
//...
}
}
By using these initialization procedures (and one, generic, remove-all-data procedure)
you can ensure that the data is in a repeatable state for each test and can easily
determine what changes have been made to the database during your tests.
Obviously, performance can be a huge factor in running these tests. The larger
and more complex the database, the longer these SetUp and TearDown methods will
take to execute. You should make sure that you are doing the minimal amount
of work in each one to accomplish the task. Specifically, there should be no
need to modify the database schema each time; just insert and remove data values
as appropriate. If they still take too long to run, isolate the database tests
into their own TestFixture, and only run it at specific times (like during the
overnight build, or on the weekends). This increases the performance of your
tests, but lengthens the time between writing the code and getting feedback
on it. See Steven Smith’s “Get Test Infected with NUnit: Unit Test
Your .NET Data Access Layer” (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspp/html/aspnet-testwithnunit.asp).
Testing With Mock Objects
Sometimes it is impossible to test a single method on a single object without
having some kind of state built up around it. Normally this occurs when the
method interacts with another object, usually by taking an object reference
as a parameter. This is not difficult to achieve if you are in control of both
objects. Imagine an application where you have written two classes, CPayPhone
and CPhoneCard. CPayPhone
exposes a method PlaceCall(CPhoneCard
phoneCard, double phoneNumber,
int minutes) which checks the current balance on the card, places the
call, and deducts the charge from the card.
To test such a method, you must create an instance of CPhoneCard to pass in,
then test values in it after the method has returned.
[Test]
public void TestPayPhone()
{
//initialize to $30
CPhoneCard phoneCard = new CPhoneCard("30.00");
CPayPhone payPhone = new CPayPhone();
payPhone.PlaceCall(ref phoneCard, 2105551212, 15);
Assert.AreEqual(22, phoneCard.Balance,
"PhoneCard balance not $22.");
}
Your local variable phoneCard
is a kind of mock object; though it is an actual instance of CPhoneCard,
your test code is in complete control over its state. You can know what the expected
difference in its values should be post method execution. The trouble comes in
when the method requires some kind of system generated object, or an instance
of a class that cannot be quickly initialized in test code.
The object is to fool the method under test into thinking that it is getting
a real instance of the expected object, but give it a forgery. In order for
the forgery to work, it has to implement the same interface as the real object.
For simple classes, you are likely to create them yourself; simply expose all
the same methods and properties, but only write code that tracks usage and allows
you to determine if the right calls were made, then expose any other methods
or properties that your test code might use to evaluate results.
Here’s a sample mock object class for CPhoneCard:
public class MockPhoneCard : IPhoneCard
{
public bool CheckBalanceCalledFirst = false;
public bool DeductCallCalled = false;
public double CostParameter = 0;
public double CheckBalance()
{
//Make note that CheckBalance was called
//before DeductCall, if true
if(!DeductCallCalled)
CheckBalanceCalledFirst = true;
//State doesn't matter, always return 30.0
return 30.0;
}
public void DeductCall(double cost)
{
//Simply note that the method was called, and store
//the value of cost
DeductCallCalled = true;
CostParameter = cost;
}
}
The test that uses the mock object would look like:
[Test]
public void TestPayPhone()
{
IPhoneCard phoneCard = new MockPhoneCard();
CPayPhone payPhone = new CPayPhone();
payPhone.PlaceCall(ref phoneCard, 2105551212, 15);
Assert.IsTrue(phoneCard.CheckBalanceCalledFirst,
"Pay phone did not check balance first.");
Assert.AreEqual(8.00, phoneCard.CostParameter,
"The call did not cost $8.00.");
}
Writing mock objects can be a time consuming process, especially if there are
many method on the desired interface or there are many classes you need to mock.
For a more dynamic solution to mock object generation, see the NMock project
at nmock.truemesh.com.
Other Considerations
When to Write Test Code
Unit tests should be written concurrently with, immediately after, or even
before (It is called Test-First Development. See Eric
Gunnerson's MSDN article.) you write the production code. Never let too
much time elapse between writing production code and the tests that validate
it. The greater the temporal separation, the more likely that you will forget
major areas of intent and purpose in your tests. Above all else, never
check production code into the team's version control system without unit tests
to demonstrate its health. The primary purpose of unit tests is to prevent risky
code from entering the system. There are several open source projects in development
that promise to measure test coverage in your codebase; check Google periodically
to see if any of them ever release.
Where to Keep Test Code
Test code can either live in the production assembly being tested or in a separate
assembly entirely. If it lives in the tested assembly, it can be mixed with
the production code or kept separate in its own files. Where you choose to keep
your testing code depends a lot on the type of project you are developing and
the nature of the final deployment scenario. For small or internal projects,
it often makes sense to keep the unit tests in the same assembly so that you
can always run the unit tests on the library no matter where it ends up. This
allows you to verify functional units on a variety of physical platforms.
For larger projects, or projects that are shipped to external customers, storing
your tests in a separate assembly might make more sense. You can ship the test
code to persons who need it, but eliminate it for everyone else making your
application's disk footprint that much smaller.
Keeping the test code separate from the production code means having a cleaner
project with less clutter. However, keeping the two codebases together means
always having access to both. Well-written unit tests are as good or better
than well-written comments for documenting code, as they clearly express the
intended uses of the different units of functionality. Keeping the code together
means keeping this valuable documentation nearby.
How Not to Break Encapsulation
A common complaint about unit testing is that you have to design classes with
a poorly encapsulated interface. Many classes call for a variety of non-public
methods in order to execute. In order to test those methods (units of functionality)
the TestFixture class has to have access to them. The simplest solution is to
make them public, thus ensuring that all test code can access the methods, and
also ensuring that everyone else can as well. This approach is bad, since it
eliminates the value of encapsulation.
A better idea is to make sure that your tests live in the same assembly and
namespace as the class you are testing. This allows you to mark those non-public
methods and values that need to be tested as internal,
a modifier that means that code within the same assembly and namespace can access
the member. For external code, though, those members might just as well have
been marked private.
Conclusion
Writing unit tests, and executing them often, gives you a constant stream of
feedback about the health of your code. Using a tool like VSNunit that integrates
with your development environment gives you the shortest possible feedback cycle
between writing code and viewing test results. Taken together, they represent
a new way of writing code that results in healthier projects and happier programmers.
References
 |
<a href="http://www.pragmaticprogrammer.com/starter_kit/ut/" target="_blank">Pragmatic Unit Testing</a>
|
 |
<a href="http://www.amazon.com/exec/obidos/tg/detail/-/0201616416/104-4455485-8223155?v=glance" target="_blank">eXtreme
Programming Explained</a>
|
 |
<a href="http://www.xprogramming.com/" target="_blank">www.xprogramming.com</a>
|
Authors
 | Justin Gehtland is a founding member of Relevance, LLC, a consultant group dedicated to elevating the practice of software development. He is the co-author of Windows Forms Programming in Visual Basic .NET (Addison Wesley, 2003) and Effective Visual Basic (Addison Wesley, 2001). Justin is an industry speaker, and instructor with DevelopMentor in the .NET curriculum. |
|