Custom Assertions

by Paul Holser

JUnit is a powerful library for writing programmer-level tests for Java code. Its small and readily comprehensible framework enables a shallow learning curve and has made it the de facto standard for programmer testing in Java.

JUnit's maintainers purposely keep its core minimal, so to augment or complement JUnit's functionality, programmers write extensions. Thankfully, JUnit's simplicity makes it easy to extend. There are several excellent JUnit extensions available, including JUnit Addons, Cactus, and EasyMock. These extensions provide new TestCase derivatives, assertions, test runners, or some combination thereof; or they use existing JUnit functionality in interesting ways.

This article explores writing custom assertions for use in JUnit, and along the way illustrates some interesting ways to work with Java arrays.

Suppose you're writing a JUnit test, and you want to assert that two arrays are "equal." Some such assertions, as it turns out, are more equal than others. For the following, assume we have two arrays of String called expectedNames and actualNames:

Whatever the context, you probably don't want this:

Java array classes do not override the equals() method [1]. They rely on their superclass's definition of equals()--namely, that of java.lang.Object. Further, JUnit's Assert.assertEquals(Object,Object) method simply tests that expected.equals(actual); it offers no overload of assertEquals() that accepts array types or handles arrays specially [2]. Thus, [a] tests only that the two variables refer to the same array in memory; i.e. it has the same effect as:

We probably intend that "equal" arrays have the same length, and equal elements at each position. No problem:

This certainly works, but I wouldn't want to have to type this over and over every time I wanted to compare arrays. "Aha, Extract Method and have your own ArrayAssert.assertEquals()!", you say [3]. That's good sense--whenever we're confronted with the potential for duplicate code, our instincts should be to factor out the duplication, into a method on an existing or brand new class. We could do that now, but before we do I want to address a few things.

As written, our array assertion shows only the first pair of mismatched elements--once an assertion fails, the test method raises an assertion exception and bails out. Maybe more mismatches lurk undetected (until the next run of the test). To see them all at once, we'd have to trap and remember assertion failures, construct some message indicating all the failures after the loop, and fail the test with the constructed message. I smell a hack. What if we wanted to see the matching elements too? Couldn't we just print the entire contents of the array? Sure, but array classes do not override toString()--we'd have to hand-roll the string representation.

Fortunately, it's easy to turn an array into something that meets our needs--just make a List out of it:

This gives exactly the desired behavior. Arrays.asList() creates and returns a derivative of List that uses the target array as its backing store. This derivative has an overridden equals() and toString(). In accordance with List.equals()'s contract, the equals() override answers true if the comparand is a List and has the same number of elements with equal elements in corresponding positions. The overridden toString() answers something like "[Manny, Moe, Jack]". So now, if [d] fails, we see a message along the lines of:

This is fine for object arrays, but there are no overloads of Arrays.asList() that accept arrays of primitives. To achieve similar results using primitive arrays, we'll need to come up with another way of turning the arrays into Lists. Here are two possible approaches:

1. Iterate over the array, adding a primitive wrapper for each element to a List.
2. Create a fixed-size derivative of List that uses a primitive array as the backing store, à la Arrays.asList().

Here's how the approaches might look:

The first approach is straightforward--PrimitiveArrays.toList() builds and returns a new ArrayList of pre-boxed primitives. This list is effectively divorced from the array; changes to one will not affect the other. The second approach is more subtle PrimitiveArrays.asList() creates and returns a derivative--an anonymous inner class, no less!--of AbstractList that uses the primitive array as its backing store. Invocations of the list's set() method change the corresponding array element. Boxing and unboxing occur "on demand" with each call to get() and set(). AbstractList provides most of the behavior, including equals() and toString(). AbstractList's overrides of modification methods like add() and retainAll() raise UnsupportedOperationException if they would actually modify the list, so effectively we have a fixed-size view on the array--precisely what Arrays.asList() offers for object arrays.

Supporting only integer arrays isn't terribly useful; we should overload PrimitiveArrays.asList() and PrimitiveArrays.toList() for arrays of all the primitive types. However, we certainly don't want to replicate all that code for the different primitive arrays. Enter java.lang.reflect.Array (ever think you'd use this?). Here's a refactored version:

Notice how java.lang.reflect.Array staves off some duplicated, ugly, cast-filled code quite nicely by performing the boxing and unboxing of primitive array elements for us. Its get() and set() methods hide the gory details. You could even "get away with" storing, say, a java.lang.Float into a double[]-backed List, as Array.set() performs the widening conversion for you.

Attempts to store a wider boxed primitive than the backing array allows--for example, a java.lang.Long versus an int[]--cause Array.set() to raise IllegalArgumentException, however. Allowing such exceptions to propagate unhindered from our List still keeps us within the bounds of List.set()'s contract: the API documentation advertises that this exception could be raised "if some aspect of the specified element prevents it from being added to this list."

It's worth noting that these implementations are far from speed demons, what with all the instantiation, boxing, and unboxing. They are likely not ready for "prime time," and you should restrict their use to test code. [4]

So now we have the ability to turn any one-dimensional array into a List for ready comparisons via assertEquals(). Now, what if we mean to compare two arrays as though they were sets, disregarding duplicate elements? Easy--make Sets from the List views into Sets, and leverage Set's notion of equality (and of string representation):

Collections, such as HashSet, typically offer a "copy constructor" that creates a collection with the same contents as another collection. HashSet's copy constructor in particular discards duplicates for us.

Or what if we want to treat the arrays as "multisets", or "bags", where the number of occurrences of the contained elements is important? Sort the arrays first, and turn each into a List:

These assertions are getting a bit long-winded for my taste. Let's go ahead with that ArrayAssert idea.

Some items of note:

• The examples of treating arrays as bags or sets are purely contrived, to drive us toward custom assertions. If ever you find yourself wanting to treat arrays in such ways in application code, ask yourself why you aren't using appropriate classes from the Java Collections Framework or extensions thereof.
• By convention, JUnit assertions provide overloads that accept a String message as a first argument. For brevity's sake, I've left such overloads out of the example. They are trivial to add. Also in the interest of brevity, I have left out the overloads for the different types of primitive array.
• ArrayAssert extends junit.framework.Assert to gain unqualified access to the assertion methods from that class--they're static methods, so if we don't extend Assert, our other choices are:
  1. to qualify the assertEquals() calls fully, or
  2. to wait for the static import feature of Java 1.5.
  Is it smelly to perform this subclassing solely to save keystrokes? Perhaps so. However, JUnit has set such a precedent already, by making TestCase extend Assert so that extensions of TestCase need not qualify all those assertion calls. I won't be pedantic here, but be aware that this use of subclassing might be considered poor form [5].
• If the private methods of Assert that trigger test failures with specifically formatted messages, such as failNotEquals(), were made protected, subclasses of Assert such as ours could use those methods too. I've written ArrayAssert as though it had such access. It's easy enough to build your own copy of JUnit that contains those modifications in source.
• assertSetEquals() and assertBagEquals() do not simply call assertEquals() with the expected and actual sets and sorted arrays, because the failures of such assertions would not show the original arrays, but the modified ones. So these assertion methods test for the appropriate equality themselves, and call failNotEquals() using List views on the original arrays as arguments.
• Where an operation would unduly mutate the arguments, e.g. the sorting of arrays in assertBagEquals(), we defensively clone() the arrays, then mutate the copies.
• It might seem kind of funny to a consumer of assertBagEquals() that she could hand it a Comparator. I think that perhaps advertising a Comparator argument here reveals too much implementation detail. Maybe if we had another way of "bag-ifying" the arrays, we wouldn't have to resort to trickery like sorting. Unfortunately, the concept of "bag" is not directly supported in the Java Collections Framework--you're left to write your own, or find someone else's implementation [6].
• Writing tests for the assertions themselves brings to light an interesting twist. The expected outcome of a failed assertion is that an assertion exception is raised. The usual idiom for a JUnit test whose expected outcome is a raised exception is illustrated by the following:

References

• Java Language Specification
• C2 Wiki on custom assertions
• Bloch, Joshua. Effective Java Programming Language Guide. Addison-Wesley, 2001.
• Jakarta Commons Collections

Endnotes

[1] JLS 10.7 shows how an array class might be defined if its declaration were written by hand.

[2] Early looks at JUnit 4 indicate that it will have special assertion methods for arrays.

[3] Or use ArrayAssert from JUnit Addons.

[4] http://developer.java.sun.com/developer/bugParade/bugs/4348595.html

[5] Along similar lines, some developers frown on implementing an interface solely to gain unqualified access to constants declared in that interface. See Effective Java. Note that with JUnit 4 there will be no need for TestCase, hence Assert's methods can be used via static import.

[6] The Jakarta Commons Collections library supports the notion of "bag" and lots of other collection-related goodies.

© Copyright 2003 Paul R. Holser, Jr. All rights reserved.

Revised December 2005 -- de-cluttering of markup, addition of JUnit-4-related content.

Revised March 2006 -- code re-formatting.