New Ant 1.6 Features for Big Projects

New Ant 1.6 Features for Big Projects
by Stefan Bodewig

Learn new features of Ant 1.6 and see how they might impact the way you structure your build process.

Published July 2005

While the 1.5.x series of Ant releases brought a lot of improvements at the task level, it didn't change the way people used Ant. With Ant 1.6, things are a bit different. Several new features have been added to support big or just complex build scenarios. But to fully leverage their power, users may need to restructure their build process a little.

This article focuses on three of the new features — <macrodef>, <import>, <subant> tasks — to show you what you may gain by using them and how they may impact the way you'll be structuring your build setup.

Macros

Most build engineers sooner or later face a situation where they have to perform the same combination of tasks but with slightly different configurations in several places. A common example is creating a web application archive with different configurations for a development, a staging and a production system.

Let's say the web application has different web deployment descriptors depending on the target system and uses a different set of JSPs as well as a different set of libraries for the development environment. The configuration information would be placed in properties and the task to create the web archive would look similar to

  <target name="war" depends="jar">
    <war destfile="${war.name}"
         webxml="${web.xml}">
      <lib refid="support-libraries"/>
      <lib file="${jar.name}"/>
      <fileset dir="${jsps}"/>
    </war>
  </target>

where support-libraries is a reference to a <fileset> defined elsewhere that points to a common set of additional libraries that is required by your application.

If you only want to create a single web archive at a time, you just need to set the properties correctly. You could load them from a properties file specific to your target for example.

Creating archives using Ant 1.5

Now, assume you wanted to create the archives for the staging and production systems at the same time to ensure that you are really packaging up the same application for both systems. With Ant 1.5 you'd probably use <antcall> to invoke the "war" target with different property settings, something like:

  <target name="production-wars">
    <antcall target="war">
      <param name="war.name" value="${staging.war.name}"/>
      <param name="web.xml" value="${staging.web.xml}"/>
    </antcall>
    <antcall target="war">
      <param name="war.name" value="${production.war.name}"/>
      <param name="web.xml" value="${production.web.xml}"/>
    </antcall>
  </target>

Of course, this assumes that both target systems will use the same jar and JSPs.

But this approach has a major drawback — it is slow. <antcall> re-parses your build file and re-runs all targets, the called target depends upon, for every invocation. In the example above the "jar" target would be run twice. Hopefully, it would do nothing on the second invocation since the "war" target depends on it.

Creating archives using Ant 1.6

With Ant 1.6 you can forget about using <antcall> for macros, instead you can create a new task by parameterizing existing tasks. The above example would thus change to:

  <macrodef name="makewar">
    <attribute name="webxml"/>
    <attribute name="destfile"/>
    <sequential>
      <war destfile="@{destfile}"
           webxml="@{webxml}">
        <lib refid="support-libraries"/>
        <lib file="${jar.name}"/>
        <fileset dir="${jsps}"/>
      </war>
    </sequential>
  </macrodef>

This defines a task named makewar that can be used like any other task. The task has two required attributes, webxml and destfile. To make an attribute optional, we'd have to provide a default value in the tasks definition. This example assumes that ${jar.name} and ${jsps} are constant during the build process and thus they still get specified as properties. Note that properties are expanded when the task is used, not where the macro is defined.

The attributes of the task get used almost exactly like properties, they get expanded via @{} instead of ${}. Unlike properties, they are mutable, i.e. their value can (and will) change with each invocation. They are also only available inside the block of your macro definition. This means that if your macro definition contains yet another macrodef'ed task, your inner macro will not see the attributes of the containing macro.

The new production-wars target would then look like:

  <target name="production-wars">
    <makewar destfile="${staging.war.name}"
             webxml="${staging.web.xml}"/>
    <makewar destfile="${production.war.name}"
             webxml="${production.web.xml}"/>
  </target>

This new code snippet not only performs a bit faster, but is also easier to read since the attribute names provide more information.

Macro tasks can also define nested elements. The nested <fileset> of the <war> task in <makewar> 's definition could be a candidate for this. Maybe the development target needs some additional files or wants to pick JSPs or resources from different places. The following adds an optional nested <morefiles> element to the <makewar> task

  <macrodef name="makewar">
    <attribute name="webxml"/>
    <attribute name="destfile"/>
    <element name="morefiles" optional="true"/>
    <sequential>
      <war destfile="@{destfile}"
           webxml="@{webxml}">
        <lib refid="support-libraries"/>
        <lib file="${jar.name}"/>
        <fileset dir="${jsps}"/>
        <morefiles/>
      </war>
    </sequential>
  </macrodef>

An invocation would look like:

  <makewar destfile="${development.war.name}"
           webxml="${development.web.xml}">
    <morefiles>
      <fileset dir="${development.resources}"/>
      <lib refid="development-support-libraries"/>
    </morefiles>
  </makewar>

This has the same effect as if the nested elements of <morefiles> had been used inside the <war> task directly.

Even though the examples so far have only shown <macrodef> wrapping a single task, it is not limited to this.

The following macro will not only create the web archive but also ensure that the directory containing the final archive exists before it tries to write to it. In a real world build file you'd probably use a setup target to do this before you invoke the task.

  <macrodef name="makewar">
    <attribute name="webxml"/>
    <attribute name="destfile"/>
    <element name="morefiles" optional="true"/>
    <sequential>
      <dirname property="@{destfile}.parent"
               file="@{destfile}"/>
      <mkdir dir="${@{destfile}.parent}"/>
      <war destfile="@{destfile}"
           webxml="@{webxml}">
        <lib refid="support-libraries"/>
        <lib file="${jar.name}"/>
        <fileset dir="${jsps}"/>
        <morefiles/>
      </war>
    </sequential>
  </macrodef>

Note two things here:

First, attributes are expanded before properties are expanded, so the construct ${@{destfile}.parent} will expand a property who's name consists of the value of the destfile attribute and a ".parent" postfix. This means you can "nest" attribute expansions into property expansions but not the other way around.

Second, this macro defines a property with a name based on an attribute's value since properties in Ant are global and immutable. A first attempt to use

      <dirname property="parent"
               file="@{destfile}"/>

instead would not lead to the desired result on the second <makewar> invocation in the "production-wars" target. The first invocation would define a new property named parent that points to the parent directory of ${staging.war.name}. The second invocation would see this property and not change its value.

It is expected that a future version of Ant will support some kind of scoped properties that are only defined during the execution of the macro. Until then using an attribute's name to construct property names is a work-around with the potential side effect of creating lots of properties.

Tip: If you look through your build files and find uses of <antcall> as a macro substitute, it is highly recommended that you evaluate converting this to real macros using macrodef. The performance impact may be significant and it may also lead to build files that are easier to read and maintain.
Import

There are several reasons to split a build file into multiple files.

   1. The file may have become too large and needs to be split into separate sections to be easier to maintain
   2. you have a certain set of functionality that is common to more than one build file and you want to share this.

Sharing common functionality/including files before Ant 1.6

Before Ant 1.6 your only option has been the XML way of entity includes, something like

  <!DOCTYPE project [
      <!ENTITY common SYSTEM "file:./common.xml">
  ]>
 
  <project name="test" default="test" basedir=".">
 
    <target name="setup">
      ...
    </target>
 
    &common;
 
    ...
 
  </project>

taken from the Ant FAQ.

This approach has two major drawbacks. You can't use an Ant property to point to the file you want to include, so you are forced to hard-code locations into your build file. And the file you want to include is only a fragment of an XML file, it may not have a single root element and thus is more difficult to maintain with XML aware tools.

Sharing common functionality/including files with Ant 1.6

Ant 1.6 ships with a new task named import that you can use now. The example above would become

  <project name="test" default="test" basedir=".">
 
    <target name="setup">
      ...
    </target>
 
    <import file="common.xml"/>
 
    ...
 
  </project>

Since it is a task, you can use all features of Ant to specify the file location. The major difference is that the imported file has to be a valid Ant build file itself and thus must have a root element named project. If you want to convert from entity includes to import, you must wrap <project> tags around the content of the imported file, Ant will then again strip them while it reads the file.

Note that the file name is resolved relative to the location of the build file by the Ant task, not the specified base directory. You won't notice any difference if you don't set project's basedir attribute or set it to ".". If you need to resolve a file against the base directory you can use a property as a work-around, something like

  <property name="common.location" location="common.xml"/>
  <import file="${common.location}"/>

The property common.location will contain the absolute path of the file common.xml and has been resolved relative to the base directory of the importing project.

With Ant 1.6 all tasks may be placed outside or inside of targets, with two exceptions. <import> must not be nested into a target and <antcall> must not be used outside of targets (since it would create an infinite loop otherwise).

But <import> can do more than just import another file.

First of all, it defines special properties named ant.file.NAME where NAME is replaced with the name attribute of the <project> tag of the file for each imported file. This property contains the absolute path of the imported file and can be used by the imported file to locate files and resources relative to its own position (as opposed to the base directory of the importing file).

This means that <project> 's name attribute has become more important in context of the <import> task. It is also used to provide alias names for the targets defined in the imported build file. If the following file is imported

<project name="share">
    <target name="setup">
      <mkdir dir="${dest}"/>
    </target>
  </project>

the importing build file can see the target either as "setup" or "share.setup". The later becomes important in context of target overrides.

Let's assume we have a build system consisting of multiple independent components each with its own build file. The build files are almost identical so we decide to move the common functionality into a shared and imported file. For simplicity let's only cover compilation of Java files and creating a JAR archive of the results. The shared file would look like

  <project name="share">
    <target name="setup" depends="set-properties">
      <mkdir dir="${dest}/classes"/>
      <mkdir dir="${dest}/lib"/>
    </target>
    <target name="compile" depends="setup">
      <javac srcdir="${src}" destdir="${dest}/classes">
        <classpath refid="compile-classpath"/>
      </javac>
    </target>
    <target name="jar" depends="compile">
      <jar destfile="${dest}/lib/${jar.name}" basedir="${dest}/classes"/>
    </target>
  </project>

This file wouldn't work as a stand-alone Ant build file since it doesn't define the "set-properties" target that "setup" depends on.

The build file for component A could then look like

  <project name="A" default="jar">
    <target name="set-properties">
      <property name="dest" location="../dest/A"/>
      <property name="src" location="src"/>
      <property name="jar.name" value="module-A.jar"/>
      <path id="compile-classpath"/>
    </target>
    <import file="../share.xml"/>
  </project>

It would just set up the proper environment and delegate the complete build logic to the imported file. Note that the build file creates an empty path as compilation CLASSPATH since it is self-contained. Module B depends on A, its build file would look like

  <project name="B" default="jar">
    <target name="set-properties">
      <property name="dest" location="../dest/B"/>
      <property name="src" location="src"/>
      <property name="jar.name" value="module-B.jar"/>
      <path id="compile-classpath">
        <pathelement location="../dest/A/module-A.jar"/>
      </path>
    </target>
    <import file="../share.xml"/>
  </project>

You'll notice that the build file is almost identical to A's and so it seems as if it should be possible to push most of the set-properties target into shared.xml as well. In fact we can do so assuming we have a consistent naming convention for the dest and src targets.

  <project name="share">
    <target name="set-properties">
      <property name="dest" location="../dest/${ant.project.name}"/>
      <property name="src" location="src"/>
      <property name="jar.name" value="module-${ant.project.name}.jar"/>
    </target>

    ... contents of first example above ...
  </project>

ant.project.name is a built-in property that contains the value of the name attribute of the outer-most <project> tag. So if the build file for module A imports share.xml it will have the value A.

Note that all files are relative to the base directory of the importing build file, so the actual value of the src property depends on the importing file.

With this, A's build file would simply become

<project name="A" default="jar">
    <path id="compile-classpath"/>
    <import file="../share.xml"/>
  </project>

And B's

  <project name="B" default="jar">
    <path id="compile-classpath">
      <pathelement location="../dest/A/module-A.jar"/>
    </path>
    <import file="../share.xml"/>
  </project>

Now assume that B adds some RMI interfaces and needs to run <rmic> after compiling the classes but before creating the jar. This is where target overrides come handy. If we define a target in the importing build file that has the same name as one in the imported build file, the importing one will be used. For example, B could use:

  <project name="B" default="jar">
    <path id="compile-classpath">
      <pathelement location="../dest/A/module-A.jar"/>
    </path>
    <import file="../share.xml"/>

    <target name="compile" depends="setup">
      <javac srcdir="${src}" destdir="${dest}/classes">
        <classpath refid="compile-classpath"/>
      </javac>
      <rmic base="${dest}/classes" includes="**/Remote*.class"/>
    </target>
  </project>

In the above example, the "compile" target would be used instead of the one in share.xml; however, this just duplicates the <javac> task from share, which is unfortunate. A better solution would be:

  <project name="B" default="jar">
    <path id="compile-classpath">
      <pathelement location="../dest/A/module-A.jar"/>
    </path>
    <import file="../share.xml"/>

    <target name="compile" depends="share.compile">
      <rmic base="${dest}/classes" includes="**/Remote*.class"/>
    </target>
  </project>

which simply makes B's "compile" run <rmic> after the original "compile" target has been used.

If we wanted to generate some Java sources (via XDoclet for example) before compilation, we could use something like

    <import file="../share.xml"/>

    <target name="compile" depends="setup,xdoclet,share.compile"/>
    <target name="xdoclet">
       .. details of XDoclet invocation omitted ..
    </target>

So you can completely override a target or enhance it by running tasks before or after the original target.

One danger must be noted here. The target override mechanism makes the importing build file depend on the name attribute used in the imported file. If anybody changes the name attribute of the imported file, the importing build file will break. The Ant development community is currently discussing a solution to this for a future version of Ant.

Tip: If you find very common structures in your build files, it may be worth to try and refactor the files into a (some) shared file(s) and use target overrides as necessary. This can make your build system more consistent and lets you reuse build logic.
Subant

In a sense, subant is two task in one since it knows two modes of operation.

If you use <subant>'s genericantfile attribute it kind of works like <antcall> invoking a target in the same build file that contains the task. Unlike <antcall>, <subant> takes a list or set of directories and will invoke the target once for each directory setting the project's base directory. This is useful if you want to perform the exact same operation in an arbitrary number of directories.

The second mode doesn't use the genericantfile attribute but takes a list or set of build files to iterate over, calling a target in each build file. This is kind of like using the <ant> task inside a loop.

The typical scenario for this second form is a build system of several modules that can be built independently but that wants a master build file to build all modules at once.

Building a master build file prior to Ant 1.6

Taking the example discussed in the import section, such a master build file would have used

  <target name="build-all">
    <ant dir="module-A" target="jar"/>
    <ant dir="module-B" target="jar"/>
  </target>

in Ant prior to Ant 1.6.

Building a master build file with Ant 1.6

With <subant> in Ant 1.6 this can be rewritten to


  <target name="build-all">
    <subant target="jar">
      <filelist dir=".">
        <file name="module-A/build.xml"/>
        <file name="module-B/build.xml"/>
      </filelist>
    </subant>
  </target>

which doesn't look like a big win since you still have to specify each sub build file individually. The situation changes if you switch to a <fileset> instead

  <target name="build-all">
    <subant target="jar">
      <fileset dir="." includes="module-*/build.xml"/>
    </subant>
  </target>

which will automatically discover the build files for all modules. If you add a module C, the target inside the master build file doesn't need to change.

But be careful. Unlike <filelist>s or <path>s (which are also supported by <subant>) <fileset>s are not ordered. In our example module-B depended on module-A so we'd need to ensure that module-A gets built first and there is no way to do that using a <fileset>.

<fileset>s are still useful if the builds are not dependent on each other at all or they don't depend on each other for a given operation. A documentation target of module-B would probably not depend on module-A at all, neither would a target that updates your sources from your SCM system.

If you want to combine the auto-discovery of build files with ordering the builds according to their interdependencies, you'll have to write a custom Ant task. The basic idea is to write a task (let's call it <buildlist> for now) that uses a <fileset>, determines the dependencies and calculates the order <subant> has to use. It then creates a <path> that contains the build files in the correct order and places a reference to this path into the project. The invocation would look like

  <target name="build-all">
    <buildlist reference="my-build-path">
      <fileset dir="." includes="module-*/build.xml"/>
    </buildlist>
    <subant target="jar">
      <buildpath refid="my-build-path"/>
    </subant>
  </target>

The hypothetical buildlist task has already been discussed on the Ant user mailing-list and bug-tracking system. Chances are good that a future version of Ant will contain such a task.

A number of new features have been added to Ant 1.6. Many of these new capabilities make build templates easy to create, structure, and customize. In particular <import> and <target> overrides. The <import>, <macrodef>, and <subant> features stand a good chance of making Ant builds highly reusable. <scriptdef> (not discussed in this article) may be interesting for people who need some scripting but don't want to write a custom task in Java.