Tutorial: Tasks using Properties, Filesets & Paths

After reading the tutorial about writing tasks [1] this tutorial explains how to get and set properties and how to use nested filesets and paths. Finally it explains how to contribute tasks to Apache Ant.

Content

The goal

The goal is to write a task, which searches in a path for a file and saves the location of that file in a property.

Build environment

We can use the buildfile from the other tutorial and modify it a little bit. That's the advantage of using properties—we can reuse nearly the whole script. :-)

<?xml version="1.0" encoding="UTF-8"?>
<project name="FindTask" basedir="." default="test">
    ...
    <target name="use.init" description="Taskdef's the Find-Task" depends="jar">
        <taskdef name="find" classname="Find" classpath="${ant.project.name}.jar"/>
    </target>

    <!-- the other use.* targets are deleted -->
    ...
</project>

The buildfile is in the archive tutorial-tasks-filesets-properties.zip [2] in /build.xml.01-propertyaccess (future version saved as *.02..., final version as build.xml; same for sources).

Property access

Our first step is to set a property to a value and print the value of that property. So our scenario would be

    <find property="test" value="test-value"/>
    <find print="test"/>

Ok, it can be rewritten with the core tasks

    <property name="test" value="test-value"/>
    <echo message="${test}"/>

but I have to start on known ground :-)

So what to do? Handling three attributes (property, value, print) and an execute method. Because this is only an introduction example I don't do much checking:

import org.apache.tools.ant.BuildException;

public class Find extends Task {

    private String property;
    private String value;
    private String print;

    public void setProperty(String property) {
        this.property = property;
    }

    // setter for value and print

    public void execute() {
        if (print != null) {
            String propValue = getProject().getProperty(print);
            log(propValue);
        } else {
            if (property == null) throw new BuildException("property not set");
            if (value    == null) throw new BuildException("value not set");
            getProject().setNewProperty(property, value);
        }
    }
}

As said in the other tutorial, the property access is done via Project instance. We get this instance via the public getProject() method which we inherit from Task (more precisely from ProjectComponent). Reading a property is done via getProperty(propertyname) (very simple, isn't it?). This property returns the value as String or null if not set.
Setting a property is ... not really difficult, but there is more than one setter. You can use the setProperty() method which will do the job as expected. But there is a golden rule in Ant: properties are immutable. And this method sets the property to the specified value—whether it has a value before that or not. So we use another way. setNewProperty() sets the property only if there is no property with that name. Otherwise a message is logged.

(By the way, a short explanation of Ant's "namespaces"—not to be confused with XML namespaces: an <antcall> creates a new space for property names. All properties from the caller are passed to the callee, but the callee can set its own properties without notice by the caller.)

There are some other setters, too (but I haven't used them, so I can't say something to them, sorry :-)

After putting our two line example from above into a target names use.simple we can call that from our test case:

import org.junit.Assert;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.apache.tools.ant.BuildFileRule;

public class FindTest {

    @Rule
    public final BuildFileRule buildRule = new BuildFileRule();

    @Before
    public void setUp() {
        configureProject("build.xml");
    }

    @Test
    public void testSimple() {
        buildRule.executeTarget("useSimple");
        Assert.assertEquals("test-value", buildRule.getLog());
    }
}

and all works fine.

Using filesets

Ant provides a common way of bundling files: the fileset. Because you are reading this tutorial I think you know them and I don't have to spend more explanations about their usage in buildfiles. Our goal is to search for a file in path. And in this step the path is simply a fileset (or more precise: a collection of filesets). So our usage would be

    <find file="ant.jar" location="location.ant-jar">
        <fileset dir="${ant.home}" includes="**/*.jar"/>
    </find>

What do we need? A task with two attributes (file, location) and nested filesets. Because we had attribute handling already explained in the example above and the handling of nested elements is described in the other tutorial, the code should be very easy:

public class Find extends Task {

    private String file;
    private String location;
    private List<FileSet> filesets = new ArrayList<>();

    public void setFile(String file) {
        this.file = file;
    }

    public void setLocation(String location) {
        this.location = location;
    }

    public void addFileset(FileSet fileset) {
        filesets.add(fileset);
    }

    public void execute() {
    }
}

Ok—that task wouldn't do very much, but we can use it in the described manner without failure. In the next step we have to implement the execute method. And before that we will implement the appropriate test cases (TDD—test driven development).

In the other tutorial we have reused the already written targets of our buildfile. Now we will configure most of the test cases via Java code (sometimes it's much easier to write a target than doing it via Java coding). What can be tested?

Maybe you find some more test cases. But this is enough for now.
For each of these points we create a testXX method.

public class FindTest {

    @Rule
    public final BuildFileRule buildRule = new BuildFileRule();

    @Rule
    public ExpectedException tried = ExpectedException.none();

    ... // constructor, setUp as above

    @Test
    public void testMissingFile() {
        tried.expect(BuildException.class);
        tried.expectMessage("file not set");
        Find find = new Find();
        find.execute();
    }

    @Test
    public void testMissingLocation() {
        tried.expect(BuildException.class);
        tried.expectMessage("location not set");
        Find find = new Find();
        find.setFile("ant.jar");
        find.execute();
    }

    @Test
    public void testMissingFileset() {
        tried.expect(BuildException.class);
        tried.expectMessage("fileset not set");
        Find find = new Find();
        find.setFile("ant.jar");
        find.setLocation("location.ant-jar");
    }

    @Test
    public void testFileNotPresent() {
        buildRule.executeTarget("testFileNotPresent");
        String result = buildRule.getProject().getProperty("location.ant-jar");
        assertNull("Property set to wrong value.", result);
    }

    @Test
    public void testFilePresent() {
        buildRule.executeTarget("testFilePresent");
        String result = buildRule.getProject().getProperty("location.ant-jar");
        assertNotNull("Property not set.", result);
        assertTrue("Wrong file found.", result.endsWith("ant.jar"));
    }
}

If we run this test class all test cases (except testFileNotPresent) fail. Now we can implement our task, so that these test cases will pass.

    protected void validate() {
        if (file == null) throw new BuildException("file not set");
        if (location == null) throw new BuildException("location not set");
        if (filesets.size() < 1) throw new BuildException("fileset not set");
    }

    public void execute() {
        validate();                                                             // 1
        String foundLocation = null;
        for (FileSet fs : filesets) {                                           // 2
            DirectoryScanner ds = fs.getDirectoryScanner(getProject());         // 3
            for (String includedFile : ds.getIncludedFiles()) {
                String filename = includedFile.replace('\\','/');               // 4
                filename = filename.substring(filename.lastIndexOf("/") + 1);
                if (foundLocation == null && file.equals(filename)) {
                    File base  = ds.getBasedir();                               // 5
                    File found = new File(base, includedFile);
                    foundLocation = found.getAbsolutePath();
                }
            }
        }
        if (foundLocation != null)                                              // 6
            getProject().setNewProperty(location, foundLocation);
    }

On //1 we check the prerequisites for our task. Doing that in a validate() method is a common way, because we separate the prerequisites from the real work. On //2 we iterate over all nested filesets. If we don't want to handle multiple filesets, the addFileset() method has to reject the further calls. We can get the result of a fileset via its DirectoryScanner like done in //3. After that we create a platform independent String representation of the file path (//4, can be done in other ways of course). We have to do the replace(), because we work with a simple string comparison. Ant itself is platform independent and can therefore run on filesystems with slash (/, e.g. Linux) or backslash (\, e.g. Windows) as path separator. Therefore we have to unify that. If we find our file, we create an absolute path representation on //5, so that we can use that information without knowing the basedir. (This is very important on use with multiple filesets, because they can have different basedirs and the return value of the directory scanner is relative to its basedir.) Finally we store the location of the file as property, if we had found one (//6).

Ok, much more easier in this simple case would be to add the file as additional include element to all filesets. But I wanted to show how to handle complex situations without being complex :-)

The test case uses the Ant property ant.home as reference. This property is set by the Launcher class which starts ant. We can use that property in our buildfiles as a build-in property [3]. But if we create a new Ant environment we have to set that value for our own. And we use the <junit> task in fork mode. Therefore we have do modify our buildfile:

    <target name="junit" description="Runs the unit tests" depends="jar">
        <delete dir="${junit.out.dir.xml}"/>
        <mkdir  dir="${junit.out.dir.xml}"/>
        <junit printsummary="yes" haltonfailure="no">
            <classpath refid="classpath.test"/>
            <sysproperty key="ant.home" value="${ant.home}"/>
            <formatter type="xml"/>
            <batchtest fork="yes" todir="${junit.out.dir.xml}">
                <fileset dir="${src.dir}" includes="**/*Test.java"/>
            </batchtest>
        </junit>
    </target>

Using nested paths

A task providing support for filesets is a very comfortable one. But there is another possibility of bundling files: the <path>. Filesets are easy if the files are all under a common base directory. But if this is not the case, you have a problem. Another disadvantage is its speed: if you have only a few files in a huge directory structure, why not use a <filelist> instead? <path>s combines these datatypes in that way that a path contains other paths, filesets, dirsets and filelists. This is why Ant-Contrib [4] <foreach> task is modified to support paths instead of filesets. So we want that, too.

Changing from fileset to path support is very easy:

Change Java code from:
    private List<FileSet> filesets = new ArrayList<>();
    public void addFileset(FileSet fileset) {
        filesets.add(fileset);
    }
to:
    private List<Path> paths = new ArrayList<>();             *1
    public void addPath(Path path) {                          *2
        paths.add(path);
    }
and build file from:
    <find file="ant.jar" location="location.ant-jar">
        <fileset dir="${ant.home}" includes="**/*.jar"/>
    </find>
to:
    <find file="ant.jar" location="location.ant-jar">
        <path>                                                *3
            <fileset dir="${ant.home}" includes="**/*.jar"/>
        </path>
    </find>

On *1 we rename only the list. It's just for better reading the source. On *2 we have to provide the right method: an addName(Type t). Therefore replace the fileset with path here. Finally we have to modify our buildfile on *3 because our task doesn't support nested filesets any longer. So we wrap the fileset inside a path.

And now we modify the test case. Oh, not very much to do :-) Renaming the testMissingFileset() (not really a must-be but better it's named like the thing it does) and update the expected-String in that method (now a path not set message is expected). The more complex test cases base on the build script. So the targets testFileNotPresent and testFilePresent have to be modified in the manner described above.

The test are finished. Now we have to adapt the task implementation. The easiest modification is in the validate() method where we change the last line to if (paths.size()<1) throw new BuildException("path not set");. In the execute() method we have a little more work. ... mmmh ... in reality it's less work, because the Path class does the whole DirectoryScanner-handling and creating-absolute-paths stuff for us. So the execute method becomes just:

    public void execute() {
        validate();
        String foundLocation = null;
        for (Path path : paths) {                                        // 1
            for (String includedFile : path.list()) {                    // 2
                String filename = includedFile.replace('\\','/');
                filename = filename.substring(filename.lastIndexOf("/") + 1);
                if (foundLocation == null && file.equals(filename)) {
                    foundLocation = includedFile;                        // 3
                }
            }
        }
        if (foundLocation != null)
            getProject().setNewProperty(location, foundLocation);
    }

Of course we have to iterate through paths on //1. On //2 and //3 we see that the Path class does the work for us: no DirectoryScanner (was at 2) and no creating of the absolute path (was at 3).

Returning a list

So far so good. But could a file be on more than one place in the path?—Of course.
And would it be good to get all of them?—It depends ...

In this section we will extend that task to support returning a list of all files. Lists as property values are not supported by Ant natively. So we have to see how other tasks use lists. The most famous task using lists is Ant-Contrib's <foreach>. All list elements are concatenated and separated with a customizable separator (default ,).

So we do the following:

<find ... delimiter=""/> ... </find>

if the delimiter is set, we will return all found files as list with that delimiter.

Therefore we have to

So we add as test case:

in the buildfile:
    <target name="test.init">
        <mkdir dir="test1/dir11/dir111"/>                             *1
        <mkdir dir="test1/dir11/dir112"/>
        ...
        <touch file="test1/dir11/dir111/test"/>
        <touch file="test1/dir11/dir111/not"/>
        ...
        <touch file="test1/dir13/dir131/not2"/>
        <touch file="test1/dir13/dir132/test"/>
        <touch file="test1/dir13/dir132/not"/>
        <touch file="test1/dir13/dir132/not2"/>
        <mkdir dir="test2"/>
        <copy todir="test2">                                          *2
            <fileset dir="test1"/>
        </copy>
    </target>

    <target name="testMultipleFiles" depends="use.init,test.init">    *3
        <find file="test" location="location.test" delimiter=";">
            <path>
                <fileset dir="test1"/>
                <fileset dir="test2"/>
            </path>
        </find>
        <delete>                                                      *4
            <fileset dir="test1"/>
            <fileset dir="test2"/>
        </delete>
    </target>
in the test class:
    public void testMultipleFiles() {
        executeTarget("testMultipleFiles");
        String result = getProject().getProperty("location.test");
        assertNotNull("Property not set.", result);
        assertTrue("Only one file found.", result.indexOf(";") > -1);
    }

Now we need a directory structure where we CAN find files with the same name in different directories. Because we can't sure to have one we create one on *1 and *2. And of course we clean up that on *4. The creation can be done inside our test target or in a separate one, which will be better for reuse later (*3).

The task implementation is modified as followed:

    private List<String> foundFiles = new ArrayList<>();
    ...
    private String delimiter = null;
    ...
    public void setDelimiter(String delim) {
        delimiter = delim;
    }
    ...
    public void execute() {
        validate();
        // find all files
        for (Path path : paths) {
            for (File includedFile : path.list()) {
                String filename = includedFile.replace('\\','/');
                filename = filename.substring(filename.lastIndexOf("/")+1);
                if (file.equals(filename) && !foundFiles.contains(includedFile)) {  // 1
                    foundFiles.add(includedFile);
                }
            }
        }

        // create the return value (list/single)
        String rv = null;
        if (!foundFiles.isEmpty()) {                                                // 2
            if (delimiter == null) {
                // only the first
                rv = foundFiles.get(0);
            } else {
                // create list
                StringBuilder list = new StringBuilder();
                for (String file : foundFiles) {                                    // 3
                    list.append(it.next());
                    if (list.length() > 0) list.append(delimiter);                  // 4
                 }
                rv = list.toString();
            }
        }

        // create the property
        if (rv != null)
            getProject().setNewProperty(location, rv);
    }

The algorithm does: finding all files, creating the return value depending on the users wish, returning the value as property. On //1 we eliminates the duplicates. //2 ensures that we create the return value only if we have found one file. On //3 we iterate over all found files and //4 ensures that the last entry has no trailing delimiter.

Ok, first searching for all files and then returning only the first one ... You can tune the performance of your own :-)

Documentation

A task is useless if the only who is able to code the buildfile is the task developer (and he only the next few weeks :-). So documentation is also very important. In which form you do that depends on your favourite. But inside Ant there is a common format and it has advantages if you use that: all task users know that form, this form is requested if you decide to contribute your task. So we will doc our task in that form.

If you have a look at the manual page of the Java task [5] you will see that it:

As a template we have:

<!DOCTYPE html>
<html lang="en">

<head>
<title>Taskname Task</title>
</head>

<body>

<h2 id="taskname">Taskname</h2>
<h3>Description</h3>
<p>Describe the task.</p>

<h3>Parameters</h3>
<table class="attr">
  <tr>
    <th scope="col">Attribute</th>
    <th scope="col">Description</th>
    <th scope="col">Required</th>
  </tr>

  do this html row for each attribute (including inherited attributes)
  <tr>
    <td>classname</td>
    <td>the Java class to execute.</td>
    <td>Either jar or classname</td>
  </tr>

</table>

<h3>Parameters specified as nested elements</h3>

Describe each nested element (including inherited)
<h4>your nested element</h4>
<p>description</p>
<p><em>since Ant 1.6</em>.</p>

<h3>Examples</h3>
<pre>
    A code sample; don't forget to escape the < of the tags with &lt;
</pre>
What should that example do?

</body>
</html>

Here is an example documentation page for our task:

<!DOCTYPE html>
<html lang="en">

<head>
<title>Find Task</title>
</head>

<body>

<h2 id="find">Find</h2>
<h3>Description</h3>
<p>Searches in a given path for a file and returns the absolute to it as property.
If delimiter is set this task returns all found locations.</p>

<h3>Parameters</h3>
<table class="attr">
  <tr>
    <th scope="col">Attribute</th>
    <th scope="col">Description</th>
    <th scope="col">Required</th>
  </tr>
  <tr>
    <td>file</td>
    <td>The name of the file to search.</td>
    <td>yes</td>
  </tr>
  <tr>
    <td>location</td>
    <td>The name of the property where to store the location</td>
    <td>yes</td>
  </tr>
  <tr>
    <td>delimiter</td>
    <td>A delimiter to use when returning the list</td>
    <td>only if the list is required</td>
  </tr>
</table>

<h3>Parameters specified as nested elements</h3>

<h4>path</h4>
<p>The path where to search the file.</p>

<h3>Examples</h3>
<pre>
<find file="ant.jar" location="loc">
    <path>
        <fileset dir="${ant.home}"/>
    <path>
</find></pre>
Searches in Ant's home directory for a file <samp>ant.jar</samp> and stores its location in
property <code>loc</code> (should be <samp>ANT_HOME/bin/ant.jar</samp>).

<pre>
<find file="ant.jar" location="loc" delimiter=";">
    <path>
        <fileset dir="C:/"/>
    <path>
</find>
<echo>ant.jar found in: ${loc}</echo></pre>
Searches in Windows C: drive for all <samp>ant.jar</samp> and stores their locations in
property <code>loc</code> delimited with <q>;</q>. (should need a long time :-)
After that it prints out the result (e.g. <samp>C:/ant-1.5.4/bin/ant.jar;C:/ant-1.6/bin/ant.jar</samp>).

</body>
</html>

Contribute the new task

If we decide to contribute our task, we should do some things:

The Ant Task Guidelines [6] support additional information on that.

Now we will check the "Checklist before submitting a new task" described in that guideline.

Package / Directories

This task does not depend on any external library. Therefore we can use this as a core task. This task contains only one class. So we can use the standard package for core tasks: org.apache.tools.ant.taskdefs. Implementations are in the directory src/main, tests in src/testcases and buildfiles for tests in src/etc/testcases.

Now we integrate our work into Ant distribution. So first we do an update of our Git tree. If not done yet, you should clone the Ant repository on GitHub[7], then create a local clone:

git clone https://github.com/your-sig/ant.git

Now we will build our Ant distribution and do a test. So we can see if there are any tests failing on our machine. (We can ignore these failing tests on later steps; Windows syntax used here—translate to UNIX if needed):

ANTREPO> build                                                    // 1
ANTREPO> set ANT_HOME=%CD%\dist                                   // 2
ANTREPO> ant test -Dtest.haltonfailure=false                      // 3

First we have to build our Ant distribution (//1). On //2 we set the ANT_HOME environment variable to the directory where the new created distribution is stored (%CD% is expanded to the current directory on Windows 2000 and later). On //3 we let Ant do all the tests (which enforced a compile of all tests) without stopping on first failure.

Next we apply our work onto Ant sources. Because we haven't modified any, this is a relatively simple step. (Because I have a local Git clone of Ant and usually contribute my work, I work on the local copy just from the beginning. The advantage: this step isn't necessary and saves a lot of work if you modify existing sources :-).

Now our modifications are done and we will retest it:

ANTREPO> build
ANTREPO> ant run-single-test                                      // 1
             -Dtestcase=org.apache.tools.ant.taskdefs.FindTest    // 2
             -Dtest.haltonfailure=false

Because we only want to test our new class, we use the target for single tests, specify the test to use and configure not to halt on the first failure—we want to see all failures of our own test (//1 + 2).

And ... oh, all tests fail: Ant could not find the task or a class this task relies upon.

Ok: in the earlier steps we told Ant to use the Find class for the <find> task (remember the <taskdef> statement in the use.init target). But now we want to introduce that task as a core task. And nobody wants to taskdef the javac, echo, ... So what to do? The answer is the src/main/.../taskdefs/default.properties. Here is the mapping between taskname and implementing class done. So we add a find=org.apache.tools.ant.taskdefs.Find as the last core task (just before the # optional tasks line). Now a second try:

ANTREPO> build                                                    // 1
ANTREPO> ant run-single-test
             -Dtestcase=org.apache.tools.ant.taskdefs.FindTest
             -Dtest.haltonfailure=false

We have to rebuild (//1) Ant because the test look in the %ANT_HOME%\lib\ant.jar (more precise: on the classpath) for the properties file. And we have only modified it in the source path. So we have to rebuild that jar. But now all tests pass and we check whether our class breaks some other tests.

ANTREPO> ant test -Dtest.haltonfailure=false

Because there are a lot of tests this step requires a little bit of time. So use the run-single-test during development and do the test only at the end (maybe sometimes during development too). We use the -Dtest.haltonfailure=false here because there could be other tests fail and we have to look into them.

This test run should show us two things: our test will run and the number of failing tests is the same as directly after git clone (without our modifications).

Apache license statement

Simply copy the license text from one the other source from the Ant source tree.

Test on JDK 5

Ant 1.10 uses Java 8 for development, but Ant 1.9 is actively maintained, too. That means that Ant code must be able to run on a JDK 5. So we have to test that. You can download older JDKs from Oracle [8].

Clean the ANT_HOME variable, delete the build, bootstrap and dist directories, and point JAVA_HOME to the JDK 5 home directory. Then create the patch with your commit, checkout 1.9.x branch in Git, apply your patch and do the build, set ANT_HOME and run ant test (like above).

Our test should pass.

Checkstyle

There are many things we have to ensure. Indentation with 4 spaces, blanks here and there, ... (all described in the Ant Task Guidelines [6] which includes the Sun code style [9]). Because there are so many things we would be happy to have a tool for do the checks. There is one: checkstyle. Checkstyle is available at Sourceforge [10] and Ant provides with the check.xml a buildfile which will do the job for us.

Download it and put the checkstyle-*-all.jar into your %USERPROFILE%\.ant\lib directory. All jar's stored there are available to Ant so you haven't to add it to you %ANT_HOME%\lib directory (this feature is available since Ant 1.6).

So we will run the tests with

ANTREPO> ant -f check.xml checkstyle htmlreport

I prefer the HTML report because there are lots of messages and we can navigate faster. Open the ANTREPO/build/reports/checkstyle/html/index.html and navigate to the Find.java. Now we see that there are some errors: missing whitespaces, unused imports, missing javadocs. So we have to do that.

Hint: start at the bottom of the file so the line numbers in the report will keep up to date and you will find the next error place much more easier without redoing the checkstyle.

After cleaning up the code according to the messages we delete the reports directory and do a second checkstyle run. Now our task isn't listed. That's fine :-)

Publish the task

Finally we publish that archive. As described in the Ant Task Guidelines [7] we can announce it on the developer mailing list, create a BugZilla entry and open a GitHub pull request. For both we need some information:

subject short description Task for finding files in a path
body more details about the path This new task looks inside a nested <path/> for occurrences of a file and stores all locations as a property. See the included manual for details.
pull request reference GitHub pull request URL https://github.com/apache/ant/pull/0

Sending an email with this information is very easy and I think I haven't to describe that. BugZilla is slightly more difficult. But the advantage is that entries will not be forgotten (a report is generated once every weekend). So I will describe the process.

First, you must have a BugZilla account. So open the BugZilla Main Page [11] and follow the link Open a new Bugzilla account [12] and the steps described there if you haven't one.

  1. From the BugZilla main page choose Enter a new bug report [13]
  2. Choose "Ant" as product
  3. Version is the last "Alpha (nightly)" (at this time 1.10)
  4. Component is "Core tasks"
  5. Platform and Severity are ok with "Other" and "Normal"
  6. Initial State is ok with "New"
  7. Same with the empty "Assigned to"
  8. It is not required to add yourself as CC, because you are the reporter and therefore will be informed on changes
  9. URL: GitHub pull request URL
  10. Summary: add the subject from the table
  11. Description: add the body from the table
  12. Then press "Commit"

Now the new task is registered in the bug database.

Resources

  1. tutorial-writing-tasks.html
  2. tutorial-tasks-filesets-properties.zip
  3. properties.html#built-in-props
  4. http://ant-contrib.sourceforge.net/
  5. Tasks/java.html
  6. https://ant.apache.org/ant_task_guidelines.html
  7. https://github.com/apache/ant
  8. https://www.oracle.com/technetwork/java/archive-139210.html
  9. https://www.oracle.com/technetwork/java/codeconvtoc-136057.html
  10. http://checkstyle.sourceforge.net/
  11. https://issues.apache.org/bugzilla/
  12. https://issues.apache.org/bugzilla/createaccount.cgi
  13. https://issues.apache.org/bugzilla/enter_bug.cgi