Sep 08
23
The idea of a continuous integration deployment pipeline has been around for several years
and is described in detail by Dave Farley and others in
this ThoughtWorks paper.
Following my recent experience with Hudson, I thought I’d show how to create a CI pipeline
using Hudson instead of Cruise Control.
There are 2 aspects of the CI deployment pipeline that Hudson can most easily help with:
- Managing binaries
- Process Gates
The ThoughtWorks paper describes a binary repository as a shared file system where release candidates are stored.
A key feature of the process is to promote the compiled binary artefacts through each stage of the pipeline
without needing to re-build them. Each time a developer makes a checkin the system
builds a new release candidate and stores it in the binary repository ready for subsequent stages to use.
Each stage in the pipeline performs a series of tests (acceptance, performance etc.) against the same set of binary artefacts.
A consequence of this approach is that the binary artefacts must be configurable for each target environment.
For example, imagine a build process that produces a web application as a WAR file
with an embedded logging configuration. It must be possible for the deployment process to automatically
override the default embedded logging configuration with an environment specific one. The level of logging required during an
acceptance test might be verbose, whereas the level of logging produced during a performance test should be minimal. The same binary artefacts
must be deployed to the acceptance test environment and the performance test environment, and each will need to use environment specific
configuration settings.
The Process Gates refer to the stages in the deployment pipeline. As a minimum, a Commit Build stage and an Acceptance Test stage
are required. If a developer commits a change that causes the the commit build to fail then the system is broken and the developer
must either fix the problem or backout the change. When the commit build passes, the built artefacts become a release candidate.
The next stage is the (automated) Acceptance Testing. When the release candidate passes this stage it may go on to
be released into production, or may go onto an (automated) Performance Testing stage.
The ultimate aim of the pipeline is to get fully tested new functionality into production as quickly and easily as possible.
I’ve recently been doing some work with Xen virtual machines and this offers the
possibility of creating a completely new (virtual) machine from scratch, using a pre-built image of the required software packages,
to create the target deployment environment for the Performance Testing stage. The production environment would then be
built and released as a virtual machine using the same image as the Performance Test stage. You can think of the VM image, including details
about memory, disk, networking and installed pre-requisites, as just another artefact that’s part of the release candidate.
Hudson provides several features that allow us to manage binary artefacts and to define dependencies between process gates.
For my project there were 2 Ant tasks used by developers interactively:
ant full - Performs a full build and unit test. ant acceptance - Performs a build, deploys the webapp, and runs selenium acceptance tests.
I mapped these tasks to Jobs in Hudson, one called [Project]_Commit and the other called [Project]_UAT.
The expectation is that other stages will be added as new Jobs following the same naming convention,
for example [Project]_PerfTest.
Hudson provides a Post-build Action: Build other projects to tell it to run another project (Job) when the
current project completes successfully. I used this on the Commit project to tell it to run the UAT project
after the commit completed. I found this approach worked better than the alternative, which is to use the
Build Trigger: Build after other projects are built. The Commit project is triggered using the Poll SCM
build trigger which checks our subversion repository.
The Commit project was also configured to Aggregate downstream test results. This means the results of the
UAT project (once it’s completed testing) are added to the Commit builds test results. This shows at a glance if the
release candidate produced by the Commit build had errors in subsequent stages in the pipeline.
Rather than create a binary repository that’s based on a shared file system, I used a feature in Hudson to allow
access to the last successful build artefacts. To make my ant acceptance test work in the Hudson environment
I changed the way it deployed the web application. In the original interactive task used by developers, it used the
output from the build task to copy a WAR file into a new Tomcat deployment. Clearly, the principles described above
require the build artefacts from the Commit build to be deployed, rather than re-building them.
My new acceptance test ant task (called ci-acceptance) uses the published lastSuccessfulBuild artefact
URL from the Commit build instead of re-building the WAR file.
Using Hudson to build a simple deployment pipeline proved to be very simple indeed. The developers have thought about
some of the use cases required for a deployment pipeline and made it incredibly easy to setup and get started quickly.
For more information about using selenium to do acceptance testing in a
continuous integration environment, see my earlier blog on
Selenium RC Testing.
