Nemerosa

Configurable acceptance tests using Spring Boot

Damien Coraboeuf -

In a previous article, I have described how you could make your acceptance tests being delivered as an executable, so they can be executed at any time in your deployment pipeline.

Here, I will focus on the selection and configuration of tests according to the environment they run in. Typical use cases are:

  • selection of test cases according to the environment
  • configuration of connection parameters

Selection of tests

There are situations where I would not want my tests to run in very environment - for example, a test which actually updates the application should not run in production.

In order to achieve this, I'll transform the tests I created previously into injected tests, which I can control using the Spring profiles.

The @AcceptanceTestSuite annotation declares an acceptance test class.

The @AcceptanceTest allows to configure a test method to run or not in a given environment.

For example:

@AcceptanceTestSuite
class ACCBrowserBasic extends AcceptanceTestClient {

    @Test
    void 'Home page is accessible'() {
        browser {
            goTo HomePage, [:]
        }
    }

    @Test
    @AcceptanceTest(excludes = "production")
    void 'Admin login'() {
        browser { browser -> loginAsAdmin(browser) }
    }

}

Now, if I run the acceptance tests with the option --context=production, the Admin login test will not be executed while the Home page is accessible test will.

How does it work?

First of all, the JUnitAcceptanceRunner defined in a previous post will be upgraded to detect automatically the tests being annotated with @AcceptanceTestSuite:

// JUnit runtime
JUnitCore junit = new JUnitCore()

// XML reporting
XMLRunListener xmlRunListener = new XMLRunListener(System.out)
junit.addListener(xmlRunListener)

// Gets all the acceptance suites
def suites = applicationContext.getBeansWithAnnotation(AcceptanceTestSuite)

// Filters on classes
suites = suites.findAll { name, bean ->
    def acceptanceTest = applicationContext.findAnnotationOnBean(name, AcceptanceTest)
    return acceptanceTest == null || config.acceptTest(acceptanceTest)
}

// Class names
def classes = suites.values().collect { it.class }

// Creates the runners
def runners = classes.collect { new AcceptanceTestRunner(it, config) }

// Running the tests
boolean ok = runners
        .collect { junit.run(it) }
        .every { it.wasSuccessful() }

// XML output
xmlRunListener.render(new File('ontrack-acceptance.xml'))

What this runner does is to get all the test suite using the @AcceptanceTestSuite annotation and to filter the tests out (using standard JUnit API) by checking the optional @AcceptanceTest annotation against the current configuration.

When having the list of acceptance test classes, we pass them to the AcceptanceTestRunner:

// AcceptanceTestRunner.groovy
...
@Override
protected void runChild(FrameworkMethod method, RunNotifier notifier) {
    def acceptanceTest = method.getAnnotation(AcceptanceTest)
    if (acceptanceTest && !config.acceptTest(acceptanceTest)) {
        notifier.fireTestIgnored(describeChild(method))
        return
    }
    // Default
    super.runChild(method, notifier)
}

This does the same thing, filtering using the optional @AcceptanceTest annotation against the current configuration, but this time, it does it at test method level.

The config object the acceptance test configuration and will be described below.

Configuration

Every parameter I want to use for my tests (connection parameters, file system, passwords, environment, etc.) can be defined using the Spring Boot configurable properties. For example:

@Component
@ConfigurationProperties(prefix = "ontrack")
class AcceptanceConfig {
    private String url = 'http://localhost:8080'
    private boolean disableSSL = false
    private String admin = 'admin'
    private Set<String> context = [] as Set
    ...
}

By injecting this object into my JUnit runner, I can use to filter my tests as described previously:

@Autowired
JUnitAcceptanceRunner(AcceptanceConfig config, ...) {
   this.config = config
   ...
}

But since my tests are also Spring components, I can also inject this class into any test to have access to the environment:

...
@Autowired
private AcceptanceConfig config
...
String url = config.url

When running my acceptance tests, I can just pass parameters along:

--ontrack.url=http://production --ontrack.context=production

Conclusion

Using the power of Spring Boot, we can easily make our executable acceptance tests easily configurable.

  1. I use the @AcceptanceTest(excludes="production") annotation on my test classes and methods, and use the --context production parameter when running the tests.
  2. I can use the default Spring Boot configuration properties to define the environment my acceptance tests are running in.

You can see the complete code in ontrack.

This article is part of series of three about continuous delivery and automated acceptance tests using Docker and Spring Boot: