Best practices


Here are some recommendations and best practices we have gathered throughout our experience and consultancies with our customers.

Add module descriptors for all your modules

In Ivy world, module descriptors are ivy files, which are basically simple xml files describing both what the module produces as artifacts and its dependencies.

It is a good practice to write or download module descriptors for all the modules involved in your development, even for your third party dependencies, and even if they don't provide such module descriptors themselves.

First, it will seem like extra work and require time. But when you have several modules using the same third party library, then you will only need to add one line to your ivy file to get this library and all its own dependencies that you really need (if you have good module descriptors in your repository, especially with the use of module configurations). It will also be very helpful when you want to upgrade a dependency. One single change in your module ivy file and you will get the updated version with its updated (or not) dependencies.

Therefore we recommend adding ivy files for all the modules in your repository. You can even enforce this rule by setting the descriptor attribute to required on your resolvers. Hence you shouldn't need to use the dependency artifact inclusion/exclusion/specification feature of Ivy, which should only be used in very specific cases.

Use your own enterprise repository

This is usually not a valid recommendation for open source projects, but for the enterprise world we strongly suggest to avoid relying on a public repository like maven ibiblio or ivyrep. Why? Well, there are a couple of reasons:
  • control
    • The main problem with these kinds of public repositories is that you don't have control over the repository. This means that if a module descriptor is broken you cannot easily fix it. Sure you can use a chain between a shared repository and the public one and put your fixed module descriptor in the shared repository so that it hides the one on the public repository, but this makes repository browsing and maintenance cumbersome.
    Even more problematic is the possible updates of the repository. We know that versions published in such repositories should be stable and not be updated, but we also frequently see that a module descriptor is buggy, or an artifact corrupted. We even see sometimes a new version published with the same name as the preceding one because the previous one was simply badly packaged. This can occur even to the best; it occurred to us with Ivy 1.2 :-) But then we decided to publish the new version with a different name, 1.2a. But if the repository manager allows such updates, this means that what worked before can break. It can thus break your build reproducibility.
  • reliability
    • The Maven repository is not particularly well known for its reliability (we often experience major slow downs or even complete failures of the site), and ivyrep is only supported by a small company (yes we are only a small company!). So slow down and site hangs occur also. And if the repository you rely on is down, this can cause major slow downs in your development or release process.
  • accuracy
    • A public repository usually contains much more than what you actually need. Is this a problem? We think so. We think that in an enterprise environment the libraries you use should step through some kind of validation process before being used in every projects of your company. And what better way to do so? Setup an enterprise repository with only the libraries you actually want to use. This will not only ensure better quality for your application dependencies, but help to have the same versions everywhere, and even help when declaring your module dependencies, if you use a tool like IvyDE, the code completion will only show relevant information about your repository, with only the libraries you actually want to see.
  • security
    • The artifacts you download from a module repository are often executable, and are thus a security concern. Imagine a hacker replacing commons-lang by another version containing a virus? If you rely on a public repository to build your software, you expose it to a security risk. You can read more about that in this Forrester article.
Note that using an enterprise repository doesn't mean you have to build it entirely by hand. Ivy features an install task which can be used to install modules from one repository to another one, so it can be used to selectively install modules from a public repository to your enterprise repository, where you will then be able to ensure control, reliability and accuracy.

Always use patterns with at least organisation and module

Ivy is very flexible and can accomodate a lot of existing repositories, using the concept of patterns. But if your repository doesn't exist yet, we strongly recommend always using the organisation and the module name in your pattern, even for a private repository where you put only your own modules (which all have the same organisation). Why? Because the Ivy listing feature relies on the token it can find in the pattern. If you have no organisation token in your pattern, Ivy won't be able to list the (only?) organisation in your repository. And this can be a problem for code completion in IvyDE, for example, but also for repository wide tasks like install or repreport.

Public ivysettings.xml with public repositories

If you create a public repository, provide a URL to the ivysettings.xml file. It's pretty easy to do, and if someone wants to leverage your repository, he will just have to load it with settings with the URL of your ivysettings.xml file, or include it in its own configuration file, which makes it really easy to combine several public repositories.

Dealing with integration versions

Very often, especially when working in a team or with several modules, you will need to rely on intermediate, non-finalized versions of your modules. These versions are what we call integration versions, because their main objective is to be integrated with other modules to make and test an application or a framework.

If you follow the continuous integration paradigm across modules, these integration versions can be produced by a continuous integration server, very frequently.

So, how can you deal with these, possibly numerous, integration versions?

There are basically two ways to deal with them, both ways being supported by Ivy:
  • use a naming convention like a special suffix
    • the idea is pretty simple, each time you publish a new integration of your module you give the same name to the version (in maven world this is for example 1.0-SNAPSHOT). The dependency manager should then be aware that this version is special because it changes over time, so that it does not trust its local cache if it already has the version, but checks the date of the version on the repository and sees if it has changed. In Ivy this is supported using the changing attribute on a dependency or by configuring the changing pattern to use for all your modules.
  • automatically create a new version for each
    • in this case you use either a build number or a timestamp to publish each new integration version with a new version name. Then you can use one of the numerous ways in Ivy to express a version constraint. Usually selecting the very latest one (using 'latest.integration' as version constraint) is enough.
So, which way is the best? As often, it depends on your context, and if one of the two was really bad it wouldn't be supported in Ivy :-)

But usually we recommend using the second one, because using a new version each time you publish a new version better fits the version identity paradigm, and can make all your builds reproducible, even integration ones. And this is interesting because it enables, with some work in your build system, the ability to introduce a mechanism to promote an integration build to a more stable status, like a milestone or a release.

Imagine you have a customer who comes on a Monday morning and asks for the latest version of your software, for testing or demonstration purposes. Obviously he needs it for the afternoon :-) Now if you have a continuous integration process and good tracking of your changes and your artifacts, it may occur to you that you are actually able to fulfill his request without needing the use of a DeLorean to give you some more time :-) But it may also occur to you that your latest version is stable enough to be used for the purpose of the customer, but was actually built a few days ago, because the very latest just broke a feature or introduced a new one you don't want to deliver. You can deliver this 'stable' integration build if you want, but rest assured that a few days, or weeks, or even months later, the customer will ask for a bug fix on this demo only version. Why? Because it's a customer, and we all know how they are :-)

So, with a build promotion feature of any build in your repository, the solution would be pretty easy: when the customer asks for the version, you not only deliver the integration build, but you also promote it to a milestone status, for example. This promotion indicates that you should keep track of this version for a long period, to be able to come back to it and create a branch if needed.

Unfortunately Ivy does not by its own allow you to have such reproducible builds out of the box, simply because Ivy is a dependency manager, not a build tool. But if you publish only versions with a distinct name and use Ivy features like versions constraint replacement during the publication or recursive delivery of modules, it can really help.

On the other hand, the main drawback of this solution is that it can produce a lot of intermediate versions, and you will have to run some cleaning scripts in your repository unless your company name starts with a G and ends with oogle :-)

Inlining dependencies or not?

With Ivy 1.4 you can resolve a dependency without even writing an ivy file. This pratice is called inlining. But what is it good for, and when should it be avoided?

Putting ivy dependencies in a separate file has the following advantages:
  • separate revision cycle
    • if your dependencies may change more often than your build, it's a good idea to separate the two, to isolate the two concepts: describing how to build / describing your project dependencies
  • possibility to publish
    • if you describe dependencies of a module which can itself be reused, you may want to use ant to publish it to a repository. In this case the publication is only possible if you have a separate ivy file
  • more flexible
    • inline dependencies can only be used to express one dependency and only one. An ivy file can be used to express much more complex dependencies
On the other hand, using inline dependencies is very useful when:
  • you want to use a custom task in your ant build
    • Without ivy you usually either copy the custom task jar in ant lib, which requires maintenance of your workstation installation, or use a manual copy or download and a taskdef with the appropriate classpath, which is better. But if you have several custom tasks, or if they have themselves dependencies, it can become cumbersome. Using Ivy with an inline dependency is an elegant way to solve this problem.
  • you want to easily deploy an application
    • If you already build your application and its modules using Ivy, it is really easy to leverage your ivy repository to download your application and all its dependencies on the local filesystem, ready to be executed. If you also put your configuration files as artifacts in your repository (maybee packaged as a zip), the whole installation process can rely on ivy, easing the automatic installation of any version of your application available in your repository!

Hire an expert

Build and dependency management is often given too low a priority in the software development world. We often see build management implemented by developers when they have time. Even if this may seem like a time and money savings in the short term, it often turns out to be a very bad choice in the long term. Building software is not a simple task, when you want to ensure automatic, tested, fully reproducible builds, releases and installations. On the other hand, once a good build system fitting your very specific needs is setup, it can then only rely on a few people with a good understanding of what is going on, with a constant quality ensured.

Therefore hiring a build and dependency expert to analyse and improve your build and release system is most of the time a very good choice.

Feedback

These best practices reflect our own experience, but we do not pretend to own the unique truth about dependency management or even Ivy use.

So feel free to comment on this page to add your own experience feedback, suggestions or opinion.