Better JUnit-based Unit Tests with NetBeans 7.4 beta Hints

In my last post, I wrote about hints provided in NetBeans 7.4 beta that improve a developer's ability to avoid nasty runtime issues with Java exception handling. In this post, I look at how two more hints provided by NetBeans 7.4 beta can be used to make unit tests more correct and more clear during unit test execution. These are the "Inconvertible parameters of Assert.assertEquals" and "Incorrect order of parameters of Assert.assertEquals" hints.

As far as I can tell from anecdotal evidence and from talking to other Java developers, JUnit remains the most widely used unit testing framework in the Java environment. Most of these JUnit users are intimately familiar with JUnit's Assert class and its many overloaded assertEquals methods. NetBeans 7.4 beta now provides two hints to make it easier to use these assertEquals methods appropriately.

Although many of the Assert.assertEquals() methods have very specific data types for the "expected" and "actual" parameters to be asserted as equal, there is a version that accepts two Objects and this means two parameters of different types that cannot possibly be considered "equal" can still be passed to that method. There is no way for the compiler to prevent that, but NetBeans 7.4 beta includes the "Inconvertible parameters of Assert.assertEquals" hint to address that particular case. Without such a hint, one is more likely to not realize the error of his or her ways until he or she runs the JUnit test and sees the failed assertion.

One of the most common issues I've run into with JUnit (and one of the reasons I love Hamcrest's fluent API) is that I just cannot seem to remember with confidence which order the parameters to the assertEquals methods are. I have a 50/50 chance of being correct by guessing. Modern Java IDEs such as NetBeans help greatly when writing new JUnit code because their method completion features will indicate that the "expected" parameter is specified first and the "actual" parameter is specified second. More commonly, this is a problem when reading code rather than when writing it because there is no method completion helping me read the code. NetBeans 7.4 beta addresses this by highlighting the situation in which I have mixed up the parameters' order via the "Incorrect order of parameters of Assert.assertEquals" hint. With that hint enabled (which it is by default), I can quickly recognize out-of-order parameters prior to runtime and even without method completion.

Both of the hints discussed above can be demonstrated in a very simple unit test class.

Portion of CalculatorTest.java

/**
 * Test Calculator.sum(int ...).
 */
@Test
public void TestSumIntegers()
{
   final Calculator calculator = new Calculator();
   Assert.assertEquals(calculator.add(1, 2, 3, 4, 5), 15);
   Assert.assertEquals("15", calculator.add(1, 2, 3, 4, 5));
}

The code the above unit test method is testing is not important for this discussion. Rather, the focus is on the use of Assert.assertEquals in two cases. Both cases as shown above are incorrect and force demonstration of the two previously discussed NetBeans hints. The first attempt at asserting two objects are equal places the parameters in the wrong order. The "expected" value (hard-coded 15) should be listed first followed by the "actual" value calculated by the method under test. The second attempt at asserting two objects are equal will always fail because the types don't match: the first parameter is a String and the second parameter is an integer. In both of these cases, the unit test code compiles without complaint. However, both assertions will ALWAYS fail when running the unit tests. In fact, these tests results may inadvertently be interpreted as problems with the code being tested until someone looks at the test failures in greater depth.

The next two screen snapshots demonstrate NetBeans 7.4 beta flagging both problematic unit testing assertion statements.

There is one caveat to note regarding the "Incorrect order of parameters of Assert.assertEquals" hint. It works well when the assertion statement is like the one shown in my example: a hard-coded expected value is provided as the "actual" value along with an obviously calculated value as the "expected" value. The next screen snapshot illustrates this point; only the statement I earlier showed is flagged by the hint and other ways of comparing actual to expected are not flagged, even when the order is incorrect.

The last shown screen snapshot demonstrates that the NetBeans hint is only able to detect an incorrect order of assertEquals parameters (should be expected before actual rather than actual before expected) in the case where those values are directly accessed in the statement (the actual calculation is performed for the first [expected] parameter and the expected hard-coded value is provided for the second [actual] parameter).

The two hints covered in this blog post make it easier to detect problems with the frequently used JUnit Assert.assertEquals methods that might not be detected until analysis of unit test run results without the hints. Although the two issues these hints warn developers about are typically fairly easy to detect and fix, detecting and fixing these problems is still more difficult and time-consuming than having the NetBeans IDE tell you they are wrong before you even run the test.

Hamcrest Containing Matchers

The Hamcrest 1.3 Javadoc documentation for the Matchers class adds more documentation for several of that class's methods than were available in Hamcrest 1.2. For example, the four overloaded contains methods have more descriptive Javadoc documentation as shown in the two comparison screen snapshots shown next.

Although one can figure out how the "contains" matchers work just by trying them out, the Javadoc in Hamcrest 1.3 makes it easier to read how they work. Most Java developers probably think of behavior like that of String.contains(CharSequence) or Collection.contains(Object) when they think of a contains() method. In other words, most Java developers probably think of "contains" as describing if the String/Collection contains the provided characters/objects among other possible characters/objects. However, for Hamcrest matchers, "contains" has a much more specific meaning. As the Hamcrest 1.3 documentation makes much clearer, the "contains" matchers are much more sensitive to number of items and order of items being passed to these methods.

My examples shown here use JUnit and Hamcrest. It is important to emphasize here that Hamcrest's JAR file must appear on the unit tests' classpath before JUnit's JAR file or else I must use the "special" JUnit JAR file built for use with the standalone Hamcrest JAR. Using either of these approaches avoids the NoSuchMethodError and other errors (suc as org.hamcrest.Matcher.describeMismatch error) resulting from mismatched versions of classes. I have written about this JUnit/Hamcrest nuance in the blog post Moving Beyond Core Hamcrest in JUnit.

The next two screen snapshots indicate the results (as shown in NetBeans 7.3) of the unit test code snippets I show later in the blog to demonstrate Hamcrest containing matchers. The tests are supposed to have some failures (7 tests passing and 4 tests failing) to make it obvious where Hamcrest matchers may not work as one expects without reading the Javadoc. The first image shows only 5 tests passing, 2 tests failing, and 4 tests causing errors. This is because I have JUnit listed before Hamcrest on the NetBeans project's "Test Libraries" classpath. The second image shows the expected results because the Hamcrest JAR occurs before the JUnit JAR in the project's "Test Libaries" classpath.

For purposes of this demonstration, I have a simple contrived class to be tested. The source code for that Main class is shown next.

Main.java

package dustin.examples;

import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

/**
 * Main class to be unit tested.
 * 
 * @author Dustin
 */
public class Main
{
   /** Uses Java 7's diamond operator. */
   private Set<String> strings = new HashSet<>();

   public Main() {}

   public boolean addString(final String newString)
   {
      return this.strings.add(newString);
   }

   public Set<String> getStrings()
   {
      return Collections.unmodifiableSet(this.strings);
   }
}

With the class to be tested shown, it is now time to look at building some JUnit-based tests with Hamcrest matchers. Specifically, the tests are to ensure that Strings added via the class's addString(String) method are in its underlying Set and accessible via the getStrings() method. The unit test methods shown next demonstrate how to use Hamcrest matchers appropriately to determine if added Strings are contained in the class's underlying Set

Using Hamcrest contains() Matcher with Single String in Set Works

   /**
    * This test will pass because there is only a single String and so it will
    * contain that single String and order will be correct by implication.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForSingleStringSoWorks()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java"));
   }

The unit test shown above passes because the Set only has one String in it and so the order and number of Strings tested with the contains matcher matches.

Using Hamcrest Contains with Same Number of Elements Works if Order Matches

   /**
    * The "contains" matcher expects exact ordering, which really means it should
    * not be used in conjunction with {@code Set}s. Typically, either this method
    * will work and the method with same name and "2" on end will not work or
    * vice versa.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForMultipleStringsNotWorks1()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java", "Groovy"));
   }

   /**
    * The "contains" matcher expects exact ordering, which really means it should
    * not be used in conjunction with {@code Set}s. Typically, either this method
    * will work and the method with same name and "1" on end will not work or
    * vice versa.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsForMultipleStringsNotWorks2()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Groovy", "Java"));
   }

The two example unit tests shown above and their resultant output of running those test as shown in the previous screen snapshot show that as long as the number of arguments to the contains() matcher are the same as the number of Strings in the collection being tested, the match may work if the elements tested are in exactly the same order as the elements in the collection. With an unordered Set, this order cannot be relied upon, so contains() is not likely to be a good matcher to use with a unit test on a Set of more than one element.

Using Hamcrest Contains with Different Number of Elements Never Works

   /**
    * Demonstrate that contains will NOT pass when there is a different number
    * of elements asked about contains than in the collection.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsNotWorksDifferentNumberElements1()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Java"));
   }

   /**
    * Demonstrate that contains will NOT pass when there is a different number
    * of elements asked about contains than in the collection even when in
    * different order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsNotWorksDifferentNumberElements2()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, contains("Groovy"));
   }

As the JUnit test results indicate, these two unit tests never pass because the number of elements being tested for in the Set is fewer than the number of elements in the Set. In other words, this proves that the contains() matcher does not test simply for a given element being in a collection: it tests for all specified elements being present and in the specified order. This might be too limiting in some cases, so now I'll move onto some other matches Hamcrest provides for determining if an element is contained in a particular collection.

Using Hamcrest's containsInAnyOrder() Matcher

The containsInAnyOrder matcher is not as strict as the contains() matcher: it allows tested elements to be in any order within the containing collection to pass.

   /**
    * Test of addString and getStrings methods of class Main using Hamcrest
    * matcher containsInAnyOrder.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrder()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultCSharp = subject.addString("C#");
      final boolean resultGroovy = subject.addString("Groovy");
      final boolean resultScala = subject.addString("Scala");
      final boolean resultClojure = subject.addString("Clojure");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java", "C#", "Groovy", "Scala", "Clojure"));
   }

   /**
    * Use containsInAnyOrder and show that order does not matter as long as
    * all entries provided are in the collection in some order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrderAgain()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java", "Groovy"));
      assertThat(strings, containsInAnyOrder("Groovy", "Java"));
   }

The two unit tests shown immediately above both pass despite the Strings being tested being provided to the containsInAnyOrder() matcher in a different order than what they could exist in for both collections. However, the less strict containsInAnyOrder() matcher still requires all elements of the containing collection to be specified to pass. The following unit test does not pass because this condition is not met.

   /**
    * This will fail because containsInAnyOrder requires all items to be matched
    * even if in different order. With only one element being tried and two
    * elements in the collection, it will still fail. In other words, order
    * does not matter with containsInAnyOrder, but all elements in the collection
    * still need to be passed to the containsInAnyOrder matcher, just not in the
    * exact same order.
    */
   @Test
   public void testAddStringAndGetStringsWithContainsInAnyOrderDiffNumberElements()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, containsInAnyOrder("Java"));
   }

Hamcrest hasItem() and hasItems() Matchers Work As Sounds

As shown in the next two unit test methods (both of which pass), the Hamcrest hasItem() (for single item) and hasItems (for multiple items) successfully tests whether a collection has the one or more than one specified items respectively without regard for order or number of specified items. This really works more like most Java developers are used to "contains" working when working with Strings and collections.

   /**
    * Demonstrate hasItem() will also work for determining a collection contains
    * a particular item.
    */
   @Test
   public void testAddStringAndGetStringsWithHasItem()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, hasItem("Groovy"));
      assertThat(strings, hasItem("Java"));
   }

   /**
    * Demonstrate that hasItems works for determining that a collection has one
    * or more items and that the number of items and the order of the items
    * is not significant in determining pass/failure.
    */
   @Test
   public void testAddStringAndGetStringsWithHasItems()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat(strings, hasItems("Groovy", "Java"));
      assertThat(strings, hasItems("Java", "Groovy"));
      assertThat(strings, hasItems("Groovy"));
      assertThat(strings, hasItems("Java"));
   }

Hamcrest isIn() Matcher Tests Containment from Other Direction

The just-discussed hasItem() and hasItems() matchers are less strict than contains() and even less strict than containsInAnyOrder() and are often what one wants when one wants to simply ensure that one or multiple items are somewhere in a collection without concern about the item's order in that collection or that other possible items are in that collection. One other way to use Hamcrest to determine the same relationship, but from the opposite perspective, is to use isIn matcher. The isIn matcher determines if an item is somewhere with the collection provided to the matcher without regard for that item's order in the collection or whether or not there are other items in that containing collection.

   /**
    * Use isIn matcher to test individual element is in provided collection.
    */
   @Test
   public void testAddStringAndGetStringsWithIsIn()
   {
      final Main subject = new Main();
      final boolean resultJava = subject.addString("Java");
      final boolean resultGroovy = subject.addString("Groovy");
      final Set<String> strings = subject.getStrings();
      assertThat("Groovy", isIn(strings));
      assertThat("Java", isIn(strings));
   }

Conclusion

Hamcrest provides a rich set of matchers that can be used to determine if specified elements reside within a specified collection. Here are important points to keep in mind when deciding to apply these and determining which to use:

• Ensure that the Hamcrest JAR is on the test classpath before the JUnit JAR.
• Use contains when you want to ensure that the collection contains all specified items and no other items and you want the collection to contain the items in the specified order.
  • Generally avoid using contains() matcher with Sets because they are unordered by nature.
• Use containsInAnyOrder matcher when you still want to strictly test for presence of exactly same items in collection as specified in test, but don't care about the order (applicable for Sets).
• Use hasItem() and hasItems() matchers to ask a collection if it contains, possibly among other unlisted items and in no particular order, the specified item or items.
• Use isIn() matcher to ask if a particular item is in the specified collection with no regard for whether other items are in that collection or what order that item is in within the containing collection.

Moving Beyond Core Hamcrest in JUnit

In the post Improving On assertEquals with JUnit and Hamcrest I introduced use of Hamcrest with JUnit. I then looked at JUnit's Built-in Hamcrest Core Matcher Support. In this post, I look at how to apply Hamcrest's non-core matchers with JUnit. These non-core matchers are NOT included with JUnit by default, but are available by including a Hamcrest JAR in the classpath.

Although JUnit's inclusion of Hamcrest core matchers makes them easier to use if one only wants to use the core matchers, this inclusion can make use of the non-core matchers more difficult and is a well-known issue.

Because the non-core Hamcrest matchers are not included with JUnit, the Hamcrest JAR needs to be downloaded. For my examples in this post, I am using hamcrest-all-1.2.jar.

The next screen snapshot indicates the problems with combining the hamcrest-all JAR with the normal JUnit library (JUnit 4.10 as provided by NetBeans 7.2 beta in my example). As the screen snapshot indicates, when the junit-4.10.jar is included in the NetBeans libraries BEFORE the hamcrest-all-1.2.jar, the previously working code (from my previous post) breaks. Both NetBeans and the command-line compiler show this breakage in this screen snapshot.

Switching the order of the test libraries so that the Hamcrest library is listed first and the JUnit JAR listed after it, makes the compiler break on the test code go away. This is shown in the next screen snapshot.

Although switching the order of the dependent libraries so that the Hamcrest JAR is included before the JUnit JAR does prevent the build problem, this is not typically a satisfactory approach. This approach is too fragile for long-term maintainability. Fortunately, there is a better approach that JUnit directly supports to deal with this issue.

A special Hamcrest-less JUnit JAR can be downloaded. The next screen snapshot shows the one I use in this example: junit-dep-4.10.jar. The -dep in the JAR name is the clue that it's Hamcrest-free. The notation next to the JAR on the download page (screen snapshot shown next) points this out as well ("Jar without hamcrest").

With the Hamcrest-free "dep" version of the JUnit JAR, I can include it in the test libraries at any point I like with relation to the Hamcrest JAR and will still be able to build the test code. This is a much more favorable approach than relying on a specific order of test libraries. The next image shows the screen snapshot of NetBeans and the command-line build being successful even with the JUnit JAR listed first.

With the appropriate libraries in use (JUnit-dep JAR and the Hamcrest "all" JAR), all of Hamcrest's matchers can be used with JUnit-based tests. Hamcrest provides numerous matchers beyond the core matches that are now bundled with JUnit. One way to get an idea of the additional matchers available is to look at the classes in the Hamcrest JAR. The following is output from running a jar tvf command against the Hamcrest JAR and removing many of the entries to leave some of the most interesting ones. The "core" matchers tend to be based on the classes in the "core" package and the non-core matchers tend to be based on the classes in all the other packages without "core" in their name.

  4029 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/AllOf.java
  3592 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/AnyOf.java
  1774 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/CombinableMatcher.java
  1754 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/DescribedAs.java
  1104 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/Every.java
  2088 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/Is.java
  1094 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsAnything.java
  2538 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsCollectionContaining.java
  1862 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsEqual.java
  2882 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsInstanceOf.java
  1175 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsNot.java
  1230 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsNull.java
   960 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/IsSame.java
   675 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/StringContains.java
   667 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/StringEndsWith.java
   678 Thu May 21 23:21:20 MDT 2009 org/hamcrest/core/StringStartsWith.java
   
  2557 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsArray.java
  1805 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsArrayContaining.java
  1883 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsArrayContainingInAnyOrder.java
  1765 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsArrayContainingInOrder.java
  1388 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsArrayWithSize.java
  1296 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsCollectionWithSize.java
   812 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsEmptyCollection.java
   866 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsEmptyIterable.java
  1086 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsIn.java
  3426 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsIterableContainingInAnyOrder.java
  3479 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsIterableContainingInOrder.java
   993 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsIterableWithSize.java
  1899 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsMapContaining.java
  1493 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsMapContainingKey.java
  1421 Thu May 21 23:21:20 MDT 2009 org/hamcrest/collection/IsMapContainingValue.java

  1380 Thu May 21 23:21:20 MDT 2009 org/hamcrest/number/IsCloseTo.java
  2878 Thu May 21 23:21:20 MDT 2009 org/hamcrest/number/OrderingComparison.java

  1082 Thu May 21 23:21:20 MDT 2009 org/hamcrest/object/HasToString.java
   918 Thu May 21 23:21:20 MDT 2009 org/hamcrest/object/IsCompatibleType.java
  2080 Thu May 21 23:21:20 MDT 2009 org/hamcrest/object/IsEventFrom.java

  1164 Thu May 21 23:21:20 MDT 2009 org/hamcrest/text/IsEmptyString.java
  1389 Thu May 21 23:21:20 MDT 2009 org/hamcrest/text/IsEqualIgnoringCase.java
  2058 Thu May 21 23:21:20 MDT 2009 org/hamcrest/text/IsEqualIgnoringWhiteSpace.java
  1300 Thu May 21 23:21:20 MDT 2009 org/hamcrest/text/StringContainsInOrder.java

  4296 Thu May 21 23:21:20 MDT 2009 org/hamcrest/xml/HasXPath.java

JUnit's providing of a JAR without Hamcrest automatically built in (the "dep" JAR) allows developers to more carefully building up their classpaths if Hamcrest matchers above and beyond the "core" matchers are desired for use with JUnit.

JUnit's Built-in Hamcrest Core Matcher Support

In the post Improving On assertEquals with JUnit and Hamcrest, I briefly discussed Hamcrest "core" matchers being "baked in" with modern versions of JUnit. In that post, I focused particularly on use of JUnit's assertThat(T, Matcher) static method coupled with the Hamcrest core is() matcher that is automatically included in later versions of JUnit. In this post, I look at additional Hamcrest "core" matchers that are bundled with recent versions of JUnit.

Two of the advantages of JUnit including Hamcrest "core" matchers out-of-the-box is that there is no need to specifically download Hamcrest and there is no need to include it explicitly on the unit test classpaths. Before looking at more of the handy Hamcrest "core" matchers, it is important to point out here that I am intentionally and repeatedly referring to "core" Hamcrest matchers because recent versions of JUnit only provide "core" (and not all) Hamcrest matchers automatically. Any Hamcrest matchers outside of the core matchers would still need to be downloaded separately and specified explicitly on the unit test classpath. One way to get an idea of what is Hamcrest "core" (and thus what matchers are available by default in recent versions of JUnit) is to look at that package's Javadoc-based API documentation:

From this JUnit-provided documentation for the org.hamcrest.core package, we see that the following matchers (with their descriptions) are available:

Class	Javadoc Class Description	Covered Here?
AllOf<T>	Calculates the logical conjunction of two matchers.	Yes
AnyOf<T>	Calculates the logical disjunction of two matchers.	Yes
DescribedAs<T>	Provides a custom description to another matcher.	Yes
Is<T>	Decorates another Matcher, retaining the behavior but allowing tests to be slightly more expressive.	Again
IsAnything<T>	A matcher that always returns true.	No
IsEqual<T>	Is the value equal to another value, as tested by the Object.equals(java.lang.Object) invokedMethod?	Yes
IsInstanceOf	Tests whether the value is an instance of a class.	Yes
IsNot<T>	Calculates the logical negation of a matcher.	Yes
IsNull<T>	Is the value null?	Yes
IsSame<T>	Is the value the same object as another value?	Yes

In my previous post demonstrating the Hamcrest is() matcher used in conjunction with JUnit's assertThat(), I used an IntegerArithmetic implementation as test fodder. I'll use that again here for demonstrating some of the other Hamcrest core matchers. For convenience, that class is reproduced below.

IntegerArithmetic.java

package dustin.examples;

/**
 * Simple class supporting integer arithmetic.
 * 
 * @author Dustin
 */
public class IntegerArithmetic
{
   /**
    * Provide the product of the provided integers.
    * 
    * @param firstInteger First integer to be multiplied.
    * @param secondInteger Second integer to be multiplied.
    * @param integers Integers to be multiplied together for a product.
    * @return Product of the provided integers.
    * @throws ArithmeticException Thrown in my product is too small or too large
    *     to be properly represented by a Java integer.
    */
   public int multiplyIntegers(
      final int firstInteger, final int secondInteger, final int ... integers)
   {
      int returnInt = firstInteger * secondInteger;
      for (final int integer : integers)
      {
         returnInt *= integer;
      }
      return returnInt;
   }
}

In the Improving On assertEquals with JUnit and Hamcrest post, I relied largely on is() to compare expected results to actual results for the integer multiplication being tested. Another option would have been to use the equalTo matcher as shown in the next code listing.

Using Hamcrest equalTo()

   /**
    * Test of multiplyIntegers method, of class IntegerArithmetic, using core
    * Hamcrest matcher equalTo.
    */
   @Test
   public void testWithJUnitHamcrestEqualTo()
   {
      final int[] integers = {4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(2, 3, integers);
      assertThat(result, equalTo(expectedResult));
   }

Although not necessary, some developers like to use is and equalTo together because it feels more fluent to them. This is the very reason for is's existence: to make use of other matchers more fluent. I often use is() by itself (implying equalTo()) as discussed in Improving On assertEquals with JUnit and Hamcrest. The next example demonstrates using is() matcher in conjunction with the equalTo matcher.

Using Hamcrest equalTo() with is()

   /**
    * Test of multiplyIntegers method, of class IntegerArithmetic, using core
    * Hamcrest matcher equalTo with "is" Matcher..
    */
   @Test
   public void testWithJUnitHamcrestEqualToAndIsMatchers()
   {
      final int[] integers = {4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(2, 3, integers);
      assertThat(result, is(equalTo(expectedResult)));
   }

The equalTo Hamcrest matcher performs a comparison similar to calling Object.equals(Object). Indeed, its comparison functionality relies on use of the underlying object's equals(Object) implementation. This means that the last two examples will pass because the numbers being compared are logically equivalent. When one wants to ensure an even greater identity equality (actually the same objects and not just the same logical content), one can use the Hamcrest sameInstance matcher as shown in the next code listing. The not matcher is also applied because the assertion will be true and the test will pass only with the "not" in place because the expected and actual results happen to NOT be the same instances!

Using Hamcrest sameInstance() with not()

   /**
    * Test of multiplyIntegers method, of class IntegerArithmetic, using core
    * Hamcrest matchers not and sameInstance.
    */
   @Test
   public void testWithJUnitHamcrestNotSameInstance()
   {
      final int[] integers = {4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(2, 3, integers);
      assertThat(result, not(sameInstance(expectedResult)));
   }

It is sometimes desirable to control the text that is output from an assertion of a failed unit test. JUnit includes the core Hamcrest matcher asDescribed() to support this. A code example of this is shown in the next listing and the output of that failed test (and corresponding assertion) is shown in the screen snapshot of the NetBeans IDE that follows the code listing.

Using Hamcrest asDescribed() with sameInstance()

   /**
    * Test of multiplyIntegers method, of class IntegerArithmetic, using core
    * Hamcrest matchers sameInstance and asDescribed. This one will assert a
    * failure so that the asDescribed can be demonstrated (don't do this with
    * your unit tests as home)!
    */
   @Test
   public void testWithJUnitHamcrestSameInstanceDescribedAs()
   {
      final int[] integers = {4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(2, 3, integers);
      assertThat(result,
                 describedAs(
                    "Not same object (different identity reference)",
                    sameInstance(expectedResult)));
   }

Use of describedAs() allowed the reporting of a more meaningful message when the associated unit test assertion failed.

I am going to now use another contrived class to help illustrate additional core Hamcrest matchers available with recent versions of JUnit. This that "needs testing" is shown next.

SetFactory.java

package dustin.examples;

import java.lang.reflect.InvocationTargetException;
import java.util.*;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * A Factory that provides an implementation of a Set interface based on
 * supplied SetType that indicates desired type of Set implementation.
 * 
 * @author Dustin
 */
public class SetFactory<T extends Object>
{
   public enum SetType
   {
      ENUM(EnumSet.class),
      HASH(HashSet.class),
      SORTED(SortedSet.class), // SortedSet is an interface, not implementation
      TREE(TreeSet.class),
      RANDOM(Set.class);       // Set is an interface, not a concrete collection

      private Class setTypeImpl = null;

      SetType(final Class newSetType)
      {
         this.setTypeImpl = newSetType;
      }

      public Class getSetImplType()
      {
         return this.setTypeImpl;
      }
   }

   private SetFactory() {}

   public static SetFactory newInstance()
   {
      return new SetFactory();
   }

   /**
    * Creates a Set using implementation corresponding to the provided Set Type
    * that has a generic parameterized type of that specified.
    * 
    * @param setType Type of Set implementation to be used.
    * @param parameterizedType Generic parameterized type for the new set.
    * @return Newly constructed Set of provided implementation type and using
    *    the specified generic parameterized type; null if either of the provided
    *    parameters is null.
    * @throws ClassCastException Thrown if the provided SetType is SetType.ENUM,
    *    but the provided parameterizedType is not an Enum.
    */
   public Set<T> createSet(
      final SetType setType, final Class<T> parameterizedType)
   {
      if (setType == null || parameterizedType == null)
      {
         return null;
      }

      Set<T> newSet = null;
      try
      {
         switch (setType)
         {
            case ENUM:
               if (parameterizedType.isEnum())
               {
                  newSet = EnumSet.noneOf((Class<Enum>)parameterizedType);
               }
               else
               {
                  throw new ClassCastException(
                       "Provided SetType of ENUM being supplied with "
                     + "parameterized type that is not an enum ["
                     + parameterizedType.getName() + "].");
               }
               break;
            case RANDOM:
               newSet = LinkedHashSet.class.newInstance();
               break;
            case SORTED:
               newSet = TreeSet.class.newInstance();
               break;
            default:
               newSet = (Set<T>) setType.getSetImplType().getConstructor().newInstance();
               break;
         }
      }
      catch (  InstantiationException
             | IllegalAccessException
             | IllegalArgumentException
             | InvocationTargetException
             | NoSuchMethodException ex)
      {
         Logger.getLogger(SetFactory.class.getName()).log(Level.SEVERE, null, ex);
      }
      return newSet;
   }
}

The contrived class whose code was just shown provides opportunities to use additional Hamcrest "core" matchers. As described above, it's possible to use all of these matches with the is matcher to improve fluency of the statement. Two useful "core" matchers are nullValue() and notNullValue(), both of which are demonstrated in the next JUnit-based code listing (and is is used in conjunction in one case).

Using Hamcrest nullValue() and notNullValue()

   /**
    * Test of createSet method, of class SetFactory, with null SetType passed.
    */
   @Test
   public void testCreateSetNullSetType()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(null, String.class);
      assertThat(strings, nullValue());
   }

   /**
    * Test of createSet method, of class SetFactory, with null parameterized type
    * passed.
    */
   @Test
   public void testCreateSetNullParameterizedType()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, null);
      assertThat(strings, is(nullValue()));
   }

   @Test
   public void testCreateTreeSetOfStringsNotNullIfValidParams()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, String.class);
      assertThat(strings, notNullValue());
   }

The Hamcrest matcher instanceOf is also useful and is demonstrated in the next code listing (one example using instanceOf by itself and one example using it in conjunction with is).

Using Hamcrest instanceOf()

   @Test
   public void testCreateTreeSetOfStringsIsTreeSet()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, String.class);
      assertThat(strings, is(instanceOf(TreeSet.class)));
   }

   @Test
   public void testCreateEnumSet()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<RoundingMode> roundingModes = factory.createSet(SetType.ENUM, RoundingMode.class);
      roundingModes.add(RoundingMode.UP);
      assertThat(roundingModes, instanceOf(EnumSet.class));
   }

Many of the Hamcrest core matchers covered so far increase fluency and readability, but I like the next two for even more reasons. The Hamcrest hasItem() matcher checks for the existence of the prescribed item in the collection and the even more useful Hamcrest hasItems() matcher checks for the existence of multiple prescribed items in the collection. It is easier to see this in code and the following code demonstrates these in action.

Using Hamcrest hasItem() and hasItems()

   @Test
   public void testCreateTreeSetOfStringsHasOneOfAddedStrings()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, String.class);
      strings.add("Tucson");
      strings.add("Arizona");
      assertThat(strings, hasItem("Tucson"));
   }

   @Test
   public void testCreateTreeSetOfStringsHasAllOfAddedStrings()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, String.class);
      strings.add("Tucson");
      strings.add("Arizona");
      assertThat(strings, hasItems("Tucson", "Arizona"));
   }

It is sometimes desirable to test the result of a certain tested method to ensure that it meets a wide variety of expectations. This is where the Hamcrest allOf matcher comes in handy. This matcher ensures that all conditions (expressed themselves as matchers) are true. This is illustrated in the following code listing, which tests with a single assert that a generated Set is not null, has two specific Strings in it, and is an instance of TreeSet.

Using Hamcrest allOf()

   @Test
   public void testCreateSetAllKindsOfGoodness()
   {
      final SetFactory factory = SetFactory.newInstance();
      final Set<String> strings = factory.createSet(SetType.TREE, String.class);
      strings.add("Tucson");
      strings.add("Arizona");
      assertThat(
         strings,
         allOf(
            notNullValue(), hasItems("Tucson", "Arizona"), instanceOf(TreeSet.class)));
   }

To demonstrate the Hamcrest core "anyOf" matcher provided out-of-the-box with newer versions of JUnit, I am going to use yet another ridiculously contrived Java class that is in need of a unit test.

Today.java

package dustin.examples;

import java.util.Calendar;
import java.util.Locale;

/**
 * Provide what day of the week today is.
 * 
 * @author Dustin
 */
public class Today
{
   /**
    * Provide the day of the week of today's date.
    * 
    * @return Integer representing today's day of the week, corresponding to
    *    static fields defined in Calendar class.
    */
   public int getTodayDayOfWeek()
   {
      return Calendar.getInstance(Locale.US).get(Calendar.DAY_OF_WEEK);
   }
}

Now I need to test that the sole method in the class above returns a valid integer representing a day of the week correctly. I'd like my test(s) to ensure that a valid integer representing a day Sunday through Saturday is returned, but the method being tested is such that it may not be the same day of the week returned on any given test run. The code listing below indicates how this can be tested with the JUnit-included Hamcrest "anyOf" matcher.

Using Hamcrest anyOf()

   /**
    * Test of getTodayDayOfWeek method, of class Today.
    */
   @Test
   public void testGetTodayDayOfWeek()
   {
      final Today instance = new Today();
      final int todayDayOfWeek = instance.getTodayDayOfWeek();
      assertThat(todayDayOfWeek,
                 describedAs(
                    "Day of week not in range.",
                    anyOf(is(Calendar.SUNDAY),
                          is(Calendar.MONDAY),
                          is(Calendar.TUESDAY),
                          is(Calendar.WEDNESDAY),
                          is(Calendar.THURSDAY),
                          is(Calendar.FRIDAY),
                          is(Calendar.SATURDAY))));
   }

While Hamcrest's allOf requires all conditions to match for the assertion to be avoided, the existence of any one condition is sufficient to ensure that anyOf doesn't lead to an assertion of a failure.

My favorite way of determining which core Hamcrest matchers are available with JUnit is to use import completion in my Java IDE. When I statically import the org.hamcrest.CoreMatchers.* package contents, all of the available matchers are displayed. I can look in the IDE to see what the * represents to see what matchers are available to me.

It is nice to have Hamcrest "core" matchers included with JUnit and this post has attempted to demonstrate the majority of these. Hamcrest offers many useful matchers outside of the "core" that are useful as well. More details on these are available in the Hamcrest Tutorial.

Improving On assertEquals with JUnit and Hamcrest

In my blog post Are Static Imports Becoming Increasingly Accepted in Java?, I discussed the increasing use of static imports in Java to make code more fluent in certain contexts. Unit testing in Java has been particularly affected by the static import and in this blog post I provide one quick example of using static imports to make more fluent unit tests that use JUnit and Hamcrest.

The next code listing is a simple IntegerArithmetic class that has one method that needs to be unit tested.

IntegerArithmetic.java

package dustin.examples;

/**
 * Simple class supporting integer arithmetic.
 * 
 * @author Dustin
 */
public class IntegerArithmetic
{
   /**
    * Provide the product of the provided integers.
    * 
    * @param integers Integers to be multiplied together for a product.
    * @return Product of the provided integers.
    * @throws ArithmeticException Thrown in my product is too small or too large
    *     to be properly represented by a Java integer.
    */
   public int multiplyIntegers(final int ... integers)
   {
      int returnInt = 1;
      for (final int integer : integers)
      {
         returnInt *= integer;
      }
      return returnInt;
   }
}

A common approach for testing one aspect of the above method is shown next.

   /**
    * Test of multipleIntegers method, of class IntegerArithmetic, using standard
    * JUnit assertEquals.
    */
   @Test
   public void testMultipleIntegersWithDefaultJUnitAssertEquals()
   {
      final int[] integers = {2, 3, 4 , 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(integers);
      assertEquals(expectedResult, result);
   }

In the fairly typical unit test example shown above, JUnit's assertEquals is called in a fluent fashion because of the static import of org.junit.Assert.* (not shown). However, recent versions of JUnit (JUnit 4.4+) have begun including Hamcrest core matchers and this allows for an even more fluent test as depicted in the next code snippet.

   /**
    * Test of multipleIntegers method, of class IntegerArithmetic, using core
    * Hamcrest matchers included with JUnit 4.x.
    */
   @Test
   public void testMultipleIntegersWithJUnitHamcrestIs()
   {
      final int[] integers = {2, 3, 4 , 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15};
      final int expectedResult = 2 * 3 * 4 * 5 * 6 * 7 * 8 * 9 * 10 * 11 * 12 *13 * 14 * 15;
      final int result = this.instance.multiplyIntegers(integers);
      assertThat(result, is(expectedResult));
   }

In this example, JUnit's assertThat (also available as part of the static import of org.junit.Assert.* since JUnit 4.4) is used in conjunction with the included Hamcrest core matcher is(). It's certainly a matter of taste, but I prefer this second approach as more readable to me. Asserting that something (the result) is something else (the expected) seems more readable and more fluent than the older approach. It can sometimes be tricky to remember whether to list the expected or actual result first when using assertEquals and the combination of assertThat and is() makes for a little less work when I write and read tests. Even a little less work, especially when multiplied by numerous tests, is welcome.

JavaOne 2010: JUnit Kung Fu: Getting More Out of Your Unit Tests

My Wednesday at JavaOne 2010 began with John Ferguson Smart's standing-room-only presentation "JUnit Kung Fu: Getting More Out of Your Unit Tests." Most of the overcapacity audience responded in the affirmative when Smart asked who uses JUnit. While the majority of the audience uses JUnit 4, some do use JUnit 3. Only a small number of individuals raised their hands when asked who uses Test-Driven Development (TDD).

Smart stated that appropriate naming of tests is a significant tool in getting the most out of JUnit-based unit tests. He mentioned that JUnit 4 enables this by allowing for annotations to specify the types of tests rather than having to use the same method name conventions that earlier versions of JUnit requires. In the slide, "What's in a name," Smart pointed out that naming tests appropriately helps express the behavior of the application being tested. Smart likes to say he doesn't write tests for his classes. Instead, classes get tested "as a side effect" for his testing of desired behaviors. Smart recommended that you don't test "how it happens," but test "what it does." If your implementation changes, your test doesn't necessarily need to change because you're only worried about outcomes and not how the outcomes are achieved. Smart talked about how appropriately named tests are more readable for people new to the tests and also provide the benefit of helping test the appropriate things (behaviors).

Smart outlined many naming tips in his slide "What's in a name" (only a subset it listed here):

1. Don't use the word "test" in your tests (use "should" instead)
2. Write your tests consistently
3. Consider tests as production code

For Unit Test Naming Tip #1, Smart stated that "should" is very common in Behavior-Driven Development (BDD) circles. Test methods should be named to provide a context. They should provide the behavior being tested and the expected outcome. I liked this tip because I find myself naming my test methods similarly, but I have always started with the word "test," followed by the method name being tested, followed by the expected behavior and outcome. Smart's recommendations reaffirm some of the things I have found through experience, but I think provide a better articulation of how to do this in a more efficient way than I've been doing.

Smart stated that tests should be written consistently. He showed two choices: "Given-When-Then" or "Arrange-Act-Assert." Smart said that he uses the classic TDD approach of writing to the inputs and outputs first and then writing the implementation.

Smart's bullet "Tests are deliverables too - respect them as such" summarized his discussion of the importance of refactoring tests just as production code is refactored. Similarly, he stated that they should be as clean and readable as the production code. One of the common difficulties associated with unit tests is keeping them maintained and consistent with production code. Smart pointed out that if we treat the unit tests like production code, this won't be seen as a negative. Further, if tests are maintained as part of production maintenance, they don't get to a sad state of disrepair. In the questions and answers portion, Smart further recommended that unit tests be reviewed in code reviews alongside the code being reviewed.

Smart spent over 20 minutes of the 60 minute presentation on test naming conventions. He pointed out at the end of that section that if there was only one thing he wanted us to get out of this presentation, it was the important of unit testing naming conventions. I appreciated the fact that his actions (devoting 1/3 of the presentation to naming conventions for unit tests) reaffirmed his words (the one thing that we should take away).

Smart transitioned from unit test naming conventions to covering the expressiveness and readability that Hamcrest brings to JUnit-based unit testing. Smart pointed out a common weakness of JUnit related to exceptions and understanding what went wrong. Hamcrest expresses why it broke much more clearly. Smart covered "home-made Hamcrest matchers" (custom Hamcrest matchers) and described creating these in "three easy steps." Neal Ford also mentioned Hamcrest in his JavaOne 2010 presentation Unit Testing That's Not So Bad: Small Things that Make a Big Difference.

Only a few people in the audience indicated that they use parameterized tests. Smart talked about how parameterized tests are useful for data-driven tests. JUnit 4.8.1 support for parameterized tests was demonstrated. JUnit creates as many instances of the class as there are rows in the associated database table. A set of results is generated that can be analyzed. Smart also talked about using Apache POI to read in data from Excel spreadsheet to use with parameterized testing. Smart referred the audience members to his blog post Data-driven Tests with JUnit and Excel (JavaLobby version) for further details.

Smart demonstrated using parameterized tests in web application testing using Selenium 2. The purpose of this demonstration was to show that parameterized tests are not limited solely to numeric calculations.

Smart next covered JUnit Rules. He specifically discussed TemporaryFolder Rule, ErrorCollector Rule, Timeout Rule, Verifier Rule, and Watchman Rule. The post JUnit 4.7 Per-Test Rules also provides useful coverage of these rules.

Smart believes that recently added JUnit Categories will be production-ready once adequate tooling is available. You currently have to run JUnit Categories using JUnit test suites (the other work-around involves "mucking around with the classpath"). Smart's Grouping Tests Using JUnit Categories talks about JUnit Categories in significantly more detail.

Parallel tests can lead to faster running of tests, especially when multiple CPUs are available (common today). Smart showed a slide that indicated how to set up parallel tests in JUnit with Maven. This requires JUnit 4.8.1 and Surefire 2.5 (Maven).

Smart recommended that those not using a mocking framework should start using a mocking framework to make unit testing easier. He suggested that those using a mocking framework other than Mockito might look at Mockito for making their testing even easier. He stated that Mockito's mocking functionality is achieved with very little code or formality. The JUnit page on Mockito has this to say about Mockito:

Java mocking is dominated by expect-run-verify libraries like EasyMock or jMock. Mockito offers simpler and more intuitive approach: you ask questions about interactions after execution. Using mockito, you can verify what you want. Using expect-run-verify libraries you often look after irrelevant interactions.

Mockito has similar syntax to EasyMock, therefore you can refactor safely. Mockito doesn't understand the notion of 'expectation'. There is only stubbing or verifications.

Like Neal Ford, Smart mentioned Infinitest. He said it used to be open source, but is now a combination of commercial/open source. The beauty of this product is that "whenever you save your file changes [your production source code], [the applicable] unit tests will be rerun."

Smart stated something I've often believed is a common weakness in unit testing. He referred to this as a "unit test trap": our tests often do test and pass what we coded (but not necessarily the behavior we wanted). Because the coder knows what he or she coded, it is not surprising that his or her tests validate that what he or she wrote as they intended.

Regarding code coverage tools, Smart stated that these are useful but should not be solely relied upon. He pointed out that these tools show what is covered, but what we really care about is what is not test covered. My interpretation of his position here is that code coverage tools are useful to make sure that a high level of test coverage is being performed, but then further analysis needs to start from there. Developers cannot simply get overconfident because they have a high level of test coverage.

Smart stated in his presentation and then reaffirmed in the questions and answers portion that private methods should not be unit tested. His position is that if a developer is uncomfortable with them enough to unit test them individually, that developer should consider refactoring them into their own testable class. For me, this fits with his overall philosophy of testing behaviors rather than testing "how." See the StackOverflow thread What's the best way of unit testing private methods? for interesting discussion that basically summarizes common arguments for and against testing private methods directly.

Smart had significant substance to cover and ran out of time (Smart quipped that we had "approximately minus 20 seconds for questions"). This is my kind of presentation! In many ways, it was like trying to drink from a fire hose, but I loved it! There are numerous ideas and frameworks he mentioned that I plan to go spend quality time investigating further. I'm especially interested in the things that both he and Neal Ford talked about.

DISCLAIMER: As with all my reviews of JavaOne 2010 sessions, this post is clearly my interpretation of what I thought was said (or what I thought I heard). Any errors or misstatements are likely mine and not the speaker's and I recommend assuming that until proven otherwise. If anyone is aware of a misquote or misstatement in this or any of my JavaOne 2010 sessions reviews, please let me know so that I can fix it.

JavaOne 2010: Unit Testing That's Not So Bad: Small Things That Make a Big Difference

I knew that the JavaOne 2010 presentation "Unit Testing That's Not That Bad: Small Things That Make a Big Difference" would be popular and so I signed up for it early and made sure I got there in the time period in which early access seating applied.  We were told that there was a line of at least seventy people waiting to get in when there appeared to be only about 15-20 seats left so I was glad that I had signed up early.

Neal Ford began the presentation by calling his own presentation the "worst named presentation at JavaOne." He went on to say that he wanted to call this updated version of his 2009 JavaOne presentation on unit tests the same name he used last year: Unit Testing That Sucks Less: Small Things Make a Big Difference.  The name was changed on him when it was placed in the schedule information.

Ford talked about how Hamcrest makes it easier to write fluent unit tests.  He also discussed Infinitest and stated that it attempts to provide near instantaneous feedback in the IDE for unit test semantics that the IDE provides for compile time checking.  Infinitest is available as a plug-in for both Eclipse and IntelliJ, but apparently not for NetBeans or JDeveloper.  The main page for Infinitest refers to this tool as "a continuous test runner for Java."

Ford introduced Unitils and called it a "Swiss army chainsaw."  He discussed its support for various popular Java frameworks. The Unitils Summary page has a similar description: "Unitils provides general assertion utilities, support for database testing, support for testing with mock objects and offers integration with Spring, Hibernate and the Java Persistence API (JPA)."  Ford talked about Unitils's reflection assertions and lenient assertions.

Ford also discussed dbUnit support for database testing with Unitils and managing data state appropriately during the tests. I agree with Ford's assertion that using a database in a unit test can be easy to start with, but scales linearly as tests get more involved.  This approach simplifies things, but there are still performance issues with getting and releasing connections that could slow tests down significantly.

Ford covered Unitils's mocking support. He stated that it's not necessarily better or worse than other Java mocking libraries, but might be of special interest to those using Unitils for other things anyway.

In introducing JTestR, Ford stated that one of the biggest points he is making in this presentation is that in 2010 a Java developer does not need to write Java tests in Java.  He specifically mentioned using JRuby and Groovy for the unit tests. Ford stated (and I agree) that Groovy is the best suited of the two languages for Java integration. However, Ford said that the best testing tools and frameworks "on Earth" are in the Ruby community and JRuby would give access to those Ruby-based testing frameworks.  As Java + Ruby, JRuby is the obvious choice if one is testing Java code with Ruby tools.

JtestR is specifically designed for testing of Java code with Ruby-based test tools.  The main JtestR page describes JtestR this way:

JtestR is a tool that will make it easier to test Java code with state of the art Ruby tools. The main project is a collection of Ruby libraries bundled together with JRuby integration so that running tests is totally painless to set up. The project also includes a background server so that the startup cost of JRuby can be avoided. Examples of Ruby libraries included are RSpec, dust,Test/Unit, mocha and ActiveSupport.

RSpec is a behavior driven development framework modeled after JBehave.

As part of his discussion of behavior driven development, Ford talked about Ruby-based Cucumber. He talked of Cucumber reaching a level of community popularity such that other projects are being built around it.

Ford broached the subject of testing private methods. He stated that people who argue that you only test the public methods that call private methods are assuming that you don't use test-driven development. A work-around is to make all methods public or package scope, but that has its own undesirable aspects. Another approach is to use reflection.  Ford covered some of the disadvantages of using reflection in this way (significant checked exception handling and other issues that occur when you "touch Java in a sensitive place").  He used this as the segue into how Groovy makes this approach easier.

Ford pointed out that Groovy essentially turns all exceptions into runtime/unchecked exceptions.  This makes for friendlier unit test code that uses reflection.  Ford pointed out that Groovy's dirty little secret is that it ignores private and so you can access the private parts directly and avoid the reflection complexity.  Ford states that this is "technically a bug" that "they've been in no hurry to fix because it's insanely useful."

Ford showed a slide with jmock syntax and said, "This really sucks."  He then showed another slide with Groovy's Java-like syntax and its support of name/value hash pairs to make it easier to mock via hash and closure code blocks.

Ford ended with some "relatively bold statements."  He called it "professionally irresponsible to ignore software testing." He stated that it is "our [software engineering] professional rigor." He also stated that it is professional irresponsible to use the most cumbersome testing tools.  He finished with the quip and slide that writing software without unit testing is like trying to barbecue and swim at the same time.

I quoted Ford directly several times because he is good at keeping his audience engaged.  This is more challenging with a large audience like this one, but he pulled it off with humor.  I also appreciated the bold statements.  Even if there may have been a little hyperbole, such enthusiasm and quotable assertions generally make for a more memorable presentation.  There weren't many people who left this session early.