Running Individual JUnit Unit Tests from Command-line Using NetBeans build.xml File

The NetBeans IDE provides JUnit integration that can be very handy when writing and running JUnit-based unit tests. However, I like to be able to do anything I might do often outside of the IDE as well as from within the IDE. In particular, there are times when I want to do things from the command-line without the need to open up the IDE.

It can be very useful to execute a single JUnit-based unit test class rather than executing the entire unit test suite. This is easy to do in the IDE itself, but it is something I also want to do from the command-line. NetBeans supports executing a single unit test class from the command-line, but I have found that you need to be aware of a few tricks to do this. This blog posting covers the minor things one needs to know to run individual JUnit-based unit tests from the command-line using an Ant build.xml file generated by NetBeans 6.1 or NetBeans 6.5 for a standard Java Application project.

When using the NetBeans New Project wizard to create a Java Application project, one gets an Ant-compliant build.xml file as the main project build file, but most of the real work is delegated to the build-impl.xml file that is generated and placed in the nbproject subdirectory of the main project working directory.

Before demonstrating how to run an individual Test Class from the command-line line, I'll first look at how to do it through the IDE. For the IDE and command-line examples, I will be using two classes two be tested and two test classes that will test those classes. The directory structure for this project with the test classes and the classes to be tested is shown next in this screen snapshot of the NetBeans "Projects" window.



The image above shows that this NetBeans project (which was created previously as a Java Application project using the NetBeans New Project creation wizard) is called "IndividualTesting." More importantly, this image shows the two main source classes (Adder and Multiplier in the "Source Packages" area) and their respective test classes (AdderTest and MultiplierTest in the "Test Packages" area). The image also shows that Java SE 6 and JUnit 4.5 are being used.

The source code for the classes (source and test) displayed above is shown next.

Adder.java (Class to be tested)

package dustin;

/**
 * Simple class to be tested that by coincidence performs addition functionality.
 *
 * @author Dustin
 */
public class Adder
{
   /** No-arguments constructor. */
   public Adder() {}

   /**
    * Sum the provided integers.
    *
    * @param augend First integer to be added.
    * @param addend Second integer to be added.
    * @param addends Remaining integers, in any, to be added.
    * @return Sum of provided integers.
    */
   public int add(final int augend, final int addend, final int ... addends)
   {
      int sum = augend + addend;
      for (final int individualAddend : addends)
      {
         sum += individualAddend;
      }
      return sum;
   }
}

Multiplier.java (Class to be tested)

package dustin;

/**
 * Simple class to be tested that by coincidence performs multiplication functionality.
 *
 * @author Dustin
 */
public class Multiplier
{
   /**
    * Multiply the provided factors.
    *
    * @param factor1 First factor to be multiplied.
    * @param factor2 Second factor to be multiplied.
    * @param factors Remaining factors to be multiplied.
    * @return Product of factors multiplication.
    */
   public int multiply(final int factor1, final int factor2, final int ... factors)
   {
      int product = factor1 * factor2;
      for (final int individualFactor : factors)
      {
          product *= individualFactor;
      }
      return product;
   }
}

AdderTest.java (Class to test Adder)

package dustin;

import org.junit.Assert;
import org.junit.Test;

/**
 * Test for class dustin.Adder.
 *
 * @author Dustin
 */
public class AdderTest extends Adder
{
   public AdderTest() {}

   @Test
   public void testAddWithTwoAddends()
   {
      final int expectedSum = 7;
      final int resultSum = add(3,4);
      Assert.assertEquals(
         "Sum of two added integers does not match expected result.",
         expectedSum, resultSum);
   }

   @Test
   public void testAddWithThreeAddends()
   {
      final int expectedSum = 11;
      final int resultSum = add(3,4,4);
      Assert.assertEquals(
         "Sum of three added integers does not match expected result.",
         expectedSum, resultSum);
   }

   @Test
   public void testAddWithFourAddends()
   {
      final int expectedSum = 14;
      final int resultSum = add(3,4,4,3);
      Assert.assertEquals(
         "Sum of four added integers does not match expected result.",
         expectedSum, resultSum);
   }

   @Test
   public void testAddWithTwoNegativeNumbers()
   {
      final int expectedSum = -10;
      final int resultSum = add(-6,-4);
      Assert.assertEquals(
         "Sum of two negative integers does not match expected result.",
         expectedSum, resultSum);
   }

   @Test
   public void testWithIntentionalError()
   {
       final int expectedSum = 27;
       final int resultSum = add(9,3);
       Assert.assertEquals(
          "The two provided numbers do not add to what was expected.",
          expectedSum, resultSum);
   }
}

MultiplierTest.java (Class to test Multiplier)

package dustin;

import org.junit.Assert;
import org.junit.Test;


/**
 * Test for class dustin.Multiplier.
 *
 * @author Dustin
 */
public class MultiplierTest extends Multiplier
{
   /** No-arguments constructor. */
   public MultiplierTest() {}

   @Test
   public void testMultiplyTwoIntegers()
   {
      final int expectedProduct = 15;
      final int resultProduct = multiply(3,5);
      Assert.assertEquals(
         "Product of multiplication of two integers does not match.",
         expectedProduct, resultProduct);
   }

   @Test
   public void testMultiplyTwoNegativeIntegers()
   {
      final int expectedProduct = 20;
      final int resultProduct = multiply(-4,-5);
      Assert.assertEquals(
         "Product of multiplication of two negative integers does not match.",
         expectedProduct, resultProduct);
   }

   @Test
   public void testMultiplyTwoMixedSignIntegers()
   {
      final int expectedProduct = -12;
      final int resultProduct = multiply(-3,4);
      Assert.assertEquals(
         "Product of multiplication of two integers of mixed sign does not match.",
         expectedProduct, resultProduct);
   }
}

With the source code written, the JUnit-based unit tests written, and the NetBeans Java Application project set up as shown above, it is almost trivial to run the unit tests. All one needs to do is use ALT+F6 or select Run->Test Project to run all the tests for that particular NetBeans project. The next two screen snapshots show how to run all of the tests and what the results look like.

Running All Project's Unit Tests



Results of All Project's Unit Tests Displayed in NetBeans



We can see that most of the tests passed, though the test that was intentionally rigged to fail (to serve our illustration needs) did lead to a report of a single test failure. We may have reason at this point to only run a single test. For example, we might want to fix a given test and only run it rather than run all the tests again. This would be an advantage in a more realistic situation where we have many more unit test classes than the two shown here and don't want to run all of them all of the time.

From NetBeans, we can run an individual test by right-clicking on that particular test result's output and selecting "Run Again." This is demonstrated for one of the successful tests and for the failed test in the next two screen snapshots.

Running Individual Test (Successful Test)



Running Individual Test (Failed Test)



When the two individual tests are re-run individually, their respective output is shown in the next two screen snapshots.

Results of Re-running Successful Individual Test



Results of Re-running Failed Individual Test



As has been demonstrated so far, it is really easy to run JUnit-based unit tests as a group or individually against the desired test method. As stated earlier, there are times when this behavior is desired from the command-line. Running the entire suite of tests is easy and is done by simply invoking the targets compile-test and test to compile the unit tests and run the unit tests respectively. The default targets for an Ant-based build of a NetBeans project are shown in the next screen snapshot.



When one wishes to run a unit test individually from the command-line, there are a few additional details to know. From looking at the build-impl.xml file generated by NetBeans (or by looking at the listed targets in the screen snapshot above), it is evident that one can invoke the test-single target to run an individual test. When one tries to do this with a command line ant test-single, the following output is experienced.



From the end of this output (that portion displayed in the above screen snapshot: "Must select some files in the IDE or set javac.includes"), it is clear that when this particular target is not executed within the NetBeans IDE, it requires a property javac.includes to be set.

An easy method to provide the javac.includes property is to pass it as a name/value pair using the -D argument passed to the ant command. For example, to provide this to run the test "testWithIntentionalError," we can do so like this:

ant -Djavac.includes=dustin\AdderTest\testWithIntentionalError single-test

Executing the above line sets the javac.includes property to the package name and method name of the individual test to be executed.

With the javac.includes property specified, we see a different result as demonstrated in the next screen snapshot. The message is another error message and is again pretty clear: "Must select some files in the IDE or set test.includes".



The same value can be specified for the test.includes property as was specified for the javac.includes property. In this case, because we want to re-run the individual "testWithIntentionalError," we would use the following command:

ant -Djavac.includes=dustin\AdderTest\testWithIntentionalError -Dtest.includes=dustin\AdderTest\testWithIntentionalError test-single

The following screen snapshot displays the end of the properly executed individual test run.



From this screen snapshot, we see the results only for the AdderTest class (test suite) we specified in the javac.includes and test.includes properties. This can be much quicker than waiting for all the test suites to run if we have a large number of them.

As with all output of NetBeans-enabled JUnit test runs, significantly more output is available in the project's build/test/results directory in a file named TEST-dustin.AdderTest.xml (XML file named after the package and test class).

The generated XML file holding the results of the JUnit-based unit tests consists of a structure that looks something like this (I added the XML comment explaining that there are typically far more properties specified in these files, but they were left out here for brevity and clarity):

<testsuite errors="0" failures="1" hostname="MARX-PC" name="dustin.AdderTest" tests="5" time="0.109" timestamp="2009-03-15T00:24:06"> 
  <properties>
    <!-- Only a select sample of property settings are shown here.  All of the
         properties associated with the NetBeans project are declared as
         name/value property pairs here in the same way that the select property
         values below are declared. -->
    <property name="ant.file.IndividualTesting-impl" value="C:\java\examples\IndividualTesting\nbproject\build-impl.xml" /> 
    <property name="libs.proguard.javadoc" value="" /> 
    <property name="ant.library.dir" value="C:\apache-ant-1.7.0-bin\apache-ant-1.7.0\lib" /> 
    <property name="libs.Spring-2-5-6.src" value="" /> 
    <property name="libs.junit.javadoc" value="C:\Program Files\NetBeans 6.5\java2\docs\junit-3.8.2-api.zip" /> 
    <property name="javac.includes" value="dustin/AdderTest.java" /> 
    <property name="user.name" value="Dustin" />
  </properties>
<testcase classname="dustin.AdderTest" name="testAddWithTwoAddends" time="0.016" /> 
  <testcase classname="dustin.AdderTest" name="testAddWithThreeAddends" time="0.0" /> 
  <testcase classname="dustin.AdderTest" name="testAddWithFourAddends" time="0.0" /> 
  <testcase classname="dustin.AdderTest" name="testAddWithTwoNegativeNumbers" time="0.0" /> 
  <testcase classname="dustin.AdderTest" name="testWithIntentionalError" time="0.0"> 
    <failure message="The two provided numbers do not add to what was expected. expected:<27> but was:<12>" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: The two provided numbers do not add to what was expected. expected:<27> but was:<12>
 at dustin.AdderTest.testWithIntentionalError(AdderTest.java:60)
</failure> 
  </testcase> 
  <system-out><![CDATA[]]></system-out> 
  <system-err><![CDATA[]]></system-err> 
</testsuite> 

As the XML sample above indicates, the full result information is available in this file. Because it is well-formed XML, there are many tools and approaches one could use to view this data. The XML data can be viewed directly in a text editor, viewed in an XML tool that will color code it and indent it appropriately, translated with XSLT to another format, processed with Java XML parsing approaches such as JAXB, or viewed/processed with many other approaches.

Because we are using Ant and JUnit, the easiest method for viewing the test results is to take advantage of the optional Ant junitreport task. This is easily added to the build.xml file as a new target ("create-unit-test-report") as shown next:

<target name="create-unit-test-report"
        description="Generate reports for executed JUnit unit tests.">
   <mkdir dir="report" />
   <junitreport todir="./report">
      <fileset dir="./build/test/results">
         <include name="TEST-*.xml"/>
      </fileset>
      <report format="frames" todir="./report/html"/>
   </junitreport>
</target>

When this target is run, output similar to that shown in the next screen snapshot is seen.




In this case, the HTML generated via XSLT transformation of the unit test XML output is available under the project's newly created report/html directory. When the index.html file in that directory is brought up in a web browser, it appears as shown in the two images that follow.





The Properties link in the bottom right corner of the web page shown in the last image can be clicked on to see the lengthy list of properties used in the NetBeans project in a more user-friendly format.

Conclusion

NetBeans makes it easy to run all units tests in a project or specific JUnit-based unit test suites and tests individually. It is not much more difficult to run individual unit test suites using the command-line as long as the javac.includes and test.includes properties are specified when Ant is used to run the test-single target. More aesthetically pleasing output can be obtained by using Ant's junitreport tag to translate the XML output to a more desirable format such as the default HTML representation. For additional details on using NetBeans with JUnit, see Writing JUnit Tests in NetBeans IDE.