JUnit

Description

This task runs tests from the JUnit testing framework. The latest version of the framework can be found at https://junit.org. This task has been tested with JUnit 3.0 up to JUnit 3.8.2; it won't work with versions prior to JUnit 3.0. It also works with JUnit 4.x, including "pure" JUnit 4 tests using only annotations and no JUnit4TestAdapter.

Note: This task depends on external libraries not included in the Apache Ant distribution. See Library Dependencies for more information.

Note: You must have junit.jar available. You can do one of:

  1. Put both junit.jar and ant-junit.jar in ANT_HOME/lib.
  2. Do not put either in ANT_HOME/lib, and instead include their locations in your CLASSPATH environment variable.
  3. Add both JARs to your classpath using -lib.
  4. Specify the locations of both JARs using a <classpath> element in a <taskdef> in the build file.
  5. Leave ant-junit.jar in its default location in ANT_HOME/lib but include junit.jar in the <classpath> passed to <junit>. Since Ant 1.7

See the FAQ for details.

Tests are defined by nested test or batchtest tags (see nested elements).

Parameters

Attribute Description Required
printsummary Print one-line statistics for each testcase. Can take the values on, off, and withOutAndErr. withOutAndErr is the same as on but also includes the output of the test as written to System.out and System.err. No; default is off
fork Run the tests in a separate JVM. No; default is off
forkmode Controls how many JVMs get created if you want to fork some tests. Possible values are perTest (the default), perBatch and once. once creates only a single JVM for all tests while perTest creates a new JVM for each TestCase class. perBatch creates a JVM for each nested <batchtest> and one collecting all nested <test>s. Note that only tests with the same settings of filtertrace, haltonerror, haltonfailure, errorproperty and failureproperty can share a JVM, so even if you set forkmode to once, Ant may have to create more than a single JVM. This attribute is ignored for tests that don't get forked into a new JVM. Since Ant 1.6.2 No; default is perTest
haltonerror Stop the build process if an error occurs during the test run. No; default is off
errorproperty The name of a property to set in the event of an error. No
haltonfailure Stop the build process if a test fails (errors are considered failures as well). No; default is off
failureproperty The name of a property to set in the event of a failure (errors are considered failures as well). No
filtertrace Filter out JUnit and Ant stack frames from error and failure stack traces. No; default is on
timeout Cancel the individual tests if they don't finish in the given time (measured in milliseconds). Ignored if fork is off. When running multiple tests inside the same JVM (see forkMode), timeout applies to the time that all tests use together, not to an individual test. No
maxmemory Maximum amount of memory to allocate to the forked JVM. Ignored if fork is off. Note: If you get java.lang.OutOfMemoryError: Java heap space in some of your tests then you need to raise the size like maxmemory=128m No
jvm The command used to invoke JVM. The command is resolved by java.lang.Runtime.exec(). No; default is java, ignored if fork is false
dir The directory in which to invoke JVM. No, ignored if fork is false
newenvironment Do not propagate the old environment when new environment variables are specified. No; default is false, ignored if fork is false
includeantruntime Implicitly add the Ant classes required to run the tests and JUnit to the classpath in forked mode. No; default is true
showoutput Send any output generated by tests to Ant's logging system as well as to the formatters. No; by default only the formatters receive the output
outputtoformatters Send any output generated by tests to the test formatters. Since Ant 1.7.0. No; default is true
tempdir Where Ant should place temporary files. Since Ant 1.6. No; default is the project's basedir
reloading Whether or not a new classloader should be instantiated for each test case.
Ignore if fork is set to true. Since Ant 1.6.
No; default is true
clonevm If set to true, then all system properties and the bootclasspath of the forked JVM will be the same as those of the JVM running Ant. since Ant 1.7 No; default is false, ignored if fork is false
logfailedtests When Ant executes multiple tests and doesn't stop on errors or failures it will log a "FAILED" message for each failing test to its logging system. If you set this option to false, the message will not be logged and you have to rely on the formatter output to find the failing tests. since Ant 1.8.0 No
enableTestListenerEvents Whether Ant should send fine grained information about the running tests to Ant's logging system at the verbose level. Such events may be used by custom test listeners to show the progress of tests.
since Ant 1.8.2Ant 1.7.0 to 1.8.1 behave as if this attribute was true by default.
No; defaults to false, can be overridden by a magic property
threads a number of threads to run the tests in.
When this attribute is specified the tests will be split arbitrarily among the threads.
Requires that the tests be forked with the perTest option to be operative.
since Ant 1.9.4
No

By using the errorproperty and failureproperty attributes, it is possible to perform setup work (such as starting an external server), execute the test, clean up, and still fail the build in the event of a failure.

The filtertrace attribute condenses error and failure stack traces before reporting them. It works with both the plain and XML formatters. It filters out any lines that begin with the following string patterns:

   "junit.framework.TestCase"
   "junit.framework.TestResult"
   "junit.framework.TestSuite"
   "junit.framework.Assert."
   "junit.swingui.TestRunner"
   "junit.awtui.TestRunner"
   "junit.textui.TestRunner"
   "java.lang.reflect.Method.invoke("
   "sun.reflect."
   "org.apache.tools.ant."
   "org.junit."
   "junit.framework.JUnit4TestAdapter"
   " more"

Parameters specified as nested elements

The <junit> task supports a nested <classpath> element that represents a path-like structure.

Since Ant 1.7, this classpath may be used to refer to junit.jar as well as your tests and the tested code.

jvmarg

If fork is true, additional parameters may be passed to the new JVM via nested <jvmarg> elements. For example:

<junit fork="yes">
  <jvmarg value="-Djava.compiler=NONE"/>
  ...
</junit>

would run the test in a JVM without JIT.

<jvmarg> allows all attributes described in Command-line Arguments.

sysproperty

Use nested <sysproperty> elements to specify system properties required by the class. These properties will be made available to JVM during the execution of the test (either Ant's JVM or the forked JVM, if fork=true). The attributes for this element are the same as for environment variables.

<junit fork="no">
  <sysproperty key="basedir" value="${basedir}"/>
  ...
</junit>

would run the test in Ant's JVM and make the basedir property available to the test.

syspropertyset

Since Ant 1.6

You can specify a set of properties to be used as system properties with syspropertysets.

env

It is possible to specify environment variables to pass to the forked JVM via nested <env> elements. For a description of the <env> element's attributes, see the description in the exec task.

Settings will be ignored if fork=false.

bootclasspath

Since Ant 1.6.

The location of bootstrap class files can be specified using this path-like structure—will be ignored if fork is false or the target JVM doesn't support it (i.e. Java 1.1).

permissions

Since Ant 1.6.

Security permissions can be revoked and granted during the execution of the class via a nested permissions element. For more information please see permissions

Settings will be ignored if fork=true.

assertions

Since Ant 1.6.

You can control enablement of Java 1.4 assertions with an <assertions> subelement.

Assertion statements are currently ignored in non-forked mode.

modulepath

Since Ant 1.9.8

The location of modules can be specified using this path-like structure.
The modulepath requires fork to be set to true.

upgrademodulepath

Since Ant 1.9.8

The location of modules that replace upgradeable modules in the runtime image can be specified using this path-like structure.
The upgrademodulepath requires fork to be set to true.

formatter

The results of the tests can be printed in different formats. Output will always be sent to a file, unless you set the usefile attribute to false. The name of the file is determined by the name of the test and can be set by the outfile attribute of <test>.

There are four predefined formatters—one prints the test results in XML format, the other emits plain text. The formatter named brief will only print detailed information for test cases that failed, while plain gives a little statistics line for all test cases. Custom formatters that need to implement org.apache.tools.ant.taskdefs.optional.junit.JUnitResultFormatter can be specified.

If you use the XML formatter, it may not include the same output that your tests have written as some characters are illegal in XML documents and will be dropped.

The fourth formatter named failure (since Ant 1.8.0) collects all failing testXXX() methods and creates a new TestCase which delegates only these failing methods. The name and the location can be specified via Java system property or Ant property ant.junit.failureCollector. The value has to point to the directory and the name of the resulting class (without suffix). It defaults to java-tmp-dir/FailedTests.

Attribute Description Required
type Use a predefined formatter (either xml, plain, brief or failure). Exactly one of these
classname Name of a custom formatter class.
extension Extension to append to the output filename. Yes, if classname has been used
usefile Boolean that determines whether output should be sent to a file. No; default is true
if Only use formatter if the named property is set. No; default is true
unless Only use formatter if the named property is not set. No; default is true

test

Defines a single test class.

Attribute Description Required
name Name of the test class. Yes
methods Comma-separated list of names of test case methods to execute. Since 1.8.2

The methods attribute can be useful in the followiang scenarios:

  • A test method has failed and you want to re-run the test method to test a fix or re-run the test under the Java debugger without having to wait for the other (possibly long running) test methods to complete.
  • One or more test methods are running slower than expected and you want to re-run them under a Java profiler (without the overhead of running the profiler whilst other test methods are being executed).

If the methods attribute is used but no test method is specified, then no test method from the suite will be executed.

No; default is to run all test methods in the suite
fork Run the tests in a separate JVM. Overrides value set in <junit>. No
haltonerror Stop the build process if an error occurs during the test run. Overrides value set in <junit>. No
errorproperty The name of a property to set in the event of an error. Overrides value set in <junit>. No
haltonfailure Stop the build process if a test fails (errors are considered failures as well). Overrides value set in <junit>. No
failureproperty The name of a property to set in the event of a failure (errors are considered failures as well). Overrides value set in <junit>. No
filtertrace Filter out JUnit and Ant stack frames from error and failure stack traces. Overrides value set in <junit>. No; default is on
todir Directory to write the reports to. No; default is the current directory
outfile Base name of the test result. The full filename is outfile.formatter. No; default is TEST-name
if Only run test if the named property is set. No
unless Only run test if the named property is not set. No
skipNonTests Do not pass any classes that do not contain JUnit tests to the test runner. This prevents non tests from appearing as test errors in test results.
Tests are identified by looking for the @Test annotation on any methods in concrete classes that don't extend junit.framework.TestCase, or for public/protected methods with names starting with test in concrete classes that extend junit.framework.TestCase. Classes marked with the JUnit 4 org.junit.runner.RunWith or org.junit.runner.Suite.SuiteClasses annotations are also passed to JUnit for execution, as is any class with a public/protected no-argument suite() method.
No; default is false

Tests can define their own formatters via nested <formatter> elements.

batchtest

Define a number of tests based on pattern matching.

batchtest collects the included resources from any number of nested resource collections. It then generates a test class name for each resource that ends in .java or .class.

Any type of resource collection is supported as a nested element, prior to Ant 1.7 only <fileset> has been supported.

Attribute Description Required
fork Run the tests in a separate JVM. Overrides value set in <junit>. No
haltonerror Stop the build process if an error occurs during the test run. Overrides value set in <junit>. No
errorproperty The name of a property to set in the event of an error. Overrides value set in <junit>. No
haltonfailure Stop the build process if a test fails (errors are considered failures as well). Overrides value set in <junit>. No
failureproperty The name of a property to set in the event of a failure (errors are considered failures as well). Overrides value set in <junit> No
filtertrace Filter out JUnit and Ant stack frames from error and failure stack traces. Overrides value set in <junit>. No; default is on
todir Directory to write the reports to. No; default is the current directory
if Only run tests if the named property is set. No
unless Only run tests if the named property is not set. No
skipNonTests Do not pass any classes that do not contain JUnit tests to the test runner. This prevents non tests from appearing as test errors in test results.
Tests are identified by looking for the @Test annotation on any methods in concrete classes that don't extend junit.framework.TestCase, or for public/protected methods with names starting with test in concrete classes that extend junit.framework.TestCase. Classes marked with the JUnit 4 org.junit.runner.RunWith or org.junit.runner.Suite.SuiteClasses annotations are also passed to JUnit for execution, as is any class with a public/protected no-argument suite() method.
No; default is false

Batch tests can define their own formatters via nested <formatter> elements.

Forked tests and tearDown()

If a forked test runs into a timeout, Ant will terminate the JVM process it has created, which probably means the test's tearDown() method will never be called. The same is true if the forked JVM crashes for some other reason.

Since Ant 1.8.0, a special formatter is distributed with Ant that tries to load the testcase that was in the forked JVM and invoke that class' tearDown() method. This formatter has the following limitations:

If the formatter recognizes an incompatible forkMode or a suite() method or fails to load the test class it will silently do nothing.

The formatter doesn't have any effect on tests that were not forked or didn't cause timeouts or JVM crashes.

To enable the formatter, add a formatter like

<formatter classname="org.apache.tools.ant.taskdefs.optional.junit.TearDownOnVmCrash"
           usefile="false"/>

to your junit task.

ant.junit.enabletestlistenerevents magic property

Since Ant 1.8.2 the enableTestListenerEvents attribute of the task controls whether fine grained logging messages will be sent to the task's verbose log. In addition to this attribute Ant will consult the property ant.junit.enabletestlistenerevents and the value of the property overrides the setting of the attribute.

This property exists so that containers running Ant that depend on the additional logging events can ensure they will be generated even if the build file disables them.

Examples

<junit>
    <test name="my.test.TestCase"/>
</junit>

Runs the test defined in my.test.TestCase in the same VM. No output will be generated unless the test fails.

<junit printsummary="yes" fork="yes" haltonfailure="yes">
    <formatter type="plain"/>
    <test name="my.test.TestCase"/>
</junit>

Runs the test defined in my.test.TestCase in a separate JVM. At the end of the test, a one-line summary will be printed. A detailed report of the test can be found in TEST-my.test.TestCase.txt. The build process will be stopped if the test fails.

<junit printsummary="yes" haltonfailure="yes">
    <classpath>
        <pathelement location="${build.tests}"/>
        <pathelement path="${java.class.path}"/>
    </classpath>

    <formatter type="plain"/>

    <test name="my.test.TestCase" haltonfailure="no" outfile="result">
        <formatter type="xml"/>
    </test>

    <batchtest fork="yes" todir="${reports.tests}">
        <fileset dir="${src.tests}">
            <include name="**/*Test*.java"/>
            <exclude name="**/AllTests.java"/>
        </fileset>
    </batchtest>
</junit>

Runs my.test.TestCase in the same JVM, ignoring the given CLASSPATH; only a warning is printed if this test fails. In addition to the plain text test results, for this test a XML result will be output to result.xml. Then, for each matching file in the directory defined for ${src.tests} a test is run in a separate JVM. If a test fails, the build process is aborted. Results are collected in files named TEST-name.txt and written to ${reports.tests}.

<target name="test">
    <property name="collector.dir" value="${build.dir}/failingTests"/>
    <property name="collector.class" value="FailedTests"/>
    <!-- Delete 'old' collector classes -->
    <delete>
        <fileset dir="${collector.dir}" includes="${collector.class}*.class"/>
    </delete>
    <!-- compile the FailedTests class if present -->
    <javac srcdir="${collector.dir}" destdir="${collector.dir}"/>
    <available file="${collector.dir}/${collector.class}.class" property="hasFailingTests"/>
    <junit haltonerror="false" haltonfailure="false">
        <sysproperty key="ant.junit.failureCollector" value="${collector.dir}/${collector.class}"/>
        <classpath>
            <pathelement location="${collector.dir}"/>
        </classpath>
        <batchtest todir="${collector.dir}" unless="hasFailingTests">
            <fileset dir="${collector.dir}" includes="**/*.java" excludes="**/${collector.class}.*"/>
            <!-- for initial creation of the FailingTests.java -->
            <formatter type="failure"/>
            <!-- I want to see something ... -->
            <formatter type="plain" usefile="false"/>
        </batchtest>
        <test name="FailedTests" if="hasFailingTests">
            <!-- update the FailingTests.java -->
            <formatter type="failure"/>
            <!-- again, I want to see something -->
            <formatter type="plain" usefile="false"/>
        </test>
    </junit>
</target>

On the first run all tests are collected via the <batchtest/> element. Its plain formatter shows the output on the console. The failure formatter creates a Java source file in ${build.dir}/failingTests/FailedTests.java which extends junit.framework.TestCase and returns from a suite() method a test suite for the failing tests.
On a second run the collector class exists and instead of the <batchtest/> the single <test/> will run. So only the failing test cases are re-run. The two nested formatters are for displaying (for the user) and for updating the collector class.

<junit fork="true"
       jvm="${platform.java}">
    <jvmarg line="--patch-module ${module.name}=${build.test.classes}"/>
    <jvmarg line="--add-modules ${module.name}"/>
    <jvmarg line="--add-reads ${module.name}=ALL-UNNAMED"/>
    <jvmarg line="--add-exports ${module.name}/my.test=ALL-UNNAMED"/>
    <classpath>
        <pathelement path="${libs.junit}"/>
    </classpath>
    <modulepath>
        <pathelement path="${modules}:${build.classes}"/>
    </modulepath>
    <formatter type="plain"/>
    <test name="my.test.TestCase"/>
</junit>

Runs my.test.TestCase as a white-box test in the forked JVM given by the platform.java property. The JUnit library is a part of an unnamed module while the tested project and required modules are on the module path. The tests do not have module-info file and are executed in the project module given by module.name property.
The --patch-module Java option executes the tests built into ${build.test.classes} in a module given by module.name property.
The --add-modules Java option enables the tested module.
The --add-reads Java option makes the unnamed module containing JUnit readable by tested module.
The --add-exports Java option makes the non-exported test package my.test accessible from the unnamed module containing JUnit.

<junit fork="true"
       jvm="${platform.java}">
    <jvmarg line="--add-modules ${test.module.name}"/>
    <jvmarg line="--add-exports ${test.module.name}/my.test=junit,ALL-UNNAMED"/>
    <modulepath>
        <pathelement path="${modules}:${build.classes}:${libs.junit}"/>
    </modulepath>
    <formatter type="plain"/>
    <test name="my.test.TestCase"/>
</junit>

Runs my.test.TestCase as a black-box test in the forked JVM given by the platform.java property. The JUnit library is used as an automatic module. The tests' module-info requires the tested module and JUnit.
The --add-modules Java option enables the test module.
The --add-exports Java option makes the non-exported test package my.test accessible from the JUnit module and Ant's test runner. Another possibility is to export the test package in the tests' module-info by exports my.test directive.