Full automation of Jenkins pipelines and versioning

Whatever the Git workflow you're using, it is likely that you have to deal with several or many branches, like features, fixes or releases, and each of this branch must be validated using a continuous integration or deployment pipeline, that might be different according to the nature of the branch. Being able to generate automatically the integration pipelines as soon as a branch pops up is a huge gain in flexibility.

On top of this, the multiplication of branches makes it awkward to manage the versions of your product or library in a file. It is much better and much more flexible to use your branch as a version.

I'll describe here how I have put those two things together for the Ontrack application.

Branch as a version

The idea is to use the branch name as a source for the version of the product, instead of storing it in a file, which we need to synchronise each time we branch.

The diagram below illustrates the principle:

For example, on a release/* branch, like release/2.1, I'm using 2.1 as version base, appending .0, .1 as patch numbers automatically as soon as the corresponding tags have been created. No need to change anything.

I'm using the versioning plug-in in order to achieve this:

plugins {
    id 'net.nemerosa.versioning' version '1.0.0'
}
allprojects {
    ...
    version = versioning.info.full
    ...
}

The versioning plug-in provides several types of information about the version, like the full version (release-2.0-ae54d7 for example), a display version (like 2.0.2), a commit, etc. Please have a look in the plug-in's documentation for the complete listing.

You can use whichever suits your needs. For example, I'm always using the full version, but for the actual publication of some artifacts and associated POM files in the Maven Central.

See the versioning plug-in for more details about getting the version information from Git (or Subversion).

Branch pipeline generation

By using a combination of the Jenkins Job DSL plug-in and a strong naming convention for the branches, I can generate automatically integration and deployment pipelines for all the branches as soon as they appear.

The principle is illustrated below:

I'm using the seed project in order to create an init job in Jenkins, which I trigger once to create a seed job for my project (in this case, an ontrack-seed job). This job will do the rest of the work.

As soon as a new branch is created, the Project seed job will create a Branch seed job for it. In turn, this job gets the Jenkins DSL defined in the branch to create the appropriate pipeline for the branch.

Although you can imagine doing anything you like when defining your Jenkins DSL for a branch, the seed project comes in handy to generate and manage the branch pipelines themselves. Once your Project seed job has been created, you just have to focus on one thing: creating the branches!

Of course, you still have to design the DSL for your project. Since the DSL is linked to a branch, it is possible to have:

1. different pipelines per branch type (for example, for a release branch, I'll have additional jobs to be able to publish and deploy in production)
2. your pipeline is versioned and you can always recreate an old version of it using the associated branch!

For an example of a complete DSL, oyu can have a look at ontrack.

Conclusion

By using both an automated versioning based on the branch (versioning) and an automated generation of the pipelines for each branch (seed), I can just just focus on branching my project to deal with my features and releases.

Note that both those projects, versioning and seed can also work with Subversion.