@RunWith(Parameterized.class) with different set of Spring profiles per parameter

[joensson]

I am putting this post here to be able to reference it from the Spring JIRA system. The code listed here is only proof of concept, it is not production ready code.

In our system we have a setup in which we want many of our tests to be run once for each set of profiles the system will be deployed with in production.

Spring test has the @ActiveProfiles annotation, but that only allows for one set of profiles to activate for the test configuration. So either we have to subclass each test and use @ActiveProfiles in each subclass - or work around the issue

Since I commute about an hour and a half each morning and afternoon I had some spare time this friday and came up with the following very rough proof of concept (inspired by the use of @Rule in https://jira.springsource.org/browse/SPR-5292)

We currently use JUnits @Parameterized runner to run each test a number of times with different testdata - so my workaround is based on reusing this runner and creating the test context manually (using the @Rule annotation and the SpringContextRule from the JIRA referenced above.).

Basically what I did was:

1. Added an extra entry in the return value of @Parameterized.Parameters of type String[] - and added the extra String[] to the constructor of our junit test classes.
2. Use a @Rule annotated field in the test classes of type SpringContextRule - using the implementation from github referred to in the JIRA by Chris Beams.
3. Created a ProfilesShouldChangeContextListener and added @TestExecutionListeners(listeners = {DependencyInjectionTestExecutionListener.class, DirtiesContextTestExecutionListener.class, TransactionalTestExecutionListener.class, ProfilesShouldChangeContextListener.class} ) to my class under test.
4. In the ProfilesShouldChangeContextListener.prepareTestIns tance implementation I check the testclass parameters for the list of profiles to activate - and compare them to the cached context' list of active profiles. If they are different
   [**] Set the "spring.profiles.active" system property to a comma separated list of the new set of profiles to activate
   [**] Mark the context as dirty so the TestContextManager knows it has to refresh the cached context.
5. Voila - Spring 3.1 does the rest.

If the profile list is the same between test runs, the cached context is reused. If the profiles are different the TestContext is refreshed with a new configuration with the profiles listed in my test as active profiles.

I know this i a very simplistic approach - but I needed something we could use directly in our test with minimum effort.

Here is the ProfilesShouldChangeContextListener implementation - I left out the part getting the list of profiles from the test class (currently it is done using reflection in ProfilesShouldChangeContextListener.profilesFromTe stActiveProfilesField, when I have more time I will do this in a nicer way):

public class ProfilesShouldChangeContextListener extends AbstractTestExecutionListener {

    @Override
    public void prepareTestInstance(TestContext testContext) throws Exception {
        ApplicationContext applicationContext = testContext.getApplicationContext();
        Environment environment = applicationContext.getEnvironment();

        String[] activeProfiles = environment.getActiveProfiles(); //appContext profile
        TreeSet<String> currentlyActiveProfiles = new TreeSet<String>(Arrays.asList(activeProfiles));

        String[] profilesFromTestClass = ProfilesFromTestActiveProfilesField(testContext.getTestInstance());
        if (environment instanceof ConfigurableEnvironment) {
            ((ConfigurableEnvironment)environment).setActiveProfiles(profilesFromTestClass);
        } else {
            System.setProperty("spring.profiles.active", StringUtils.join(profilesFromTestClass, ","));
        }

        TreeSet<String> testInstanceSortedProfiles = new TreeSet(Arrays.asList(profilesFromTestClass));

        if (!compare(testInstanceSortedProfiles, currentlyActiveProfiles)) {
            logger.info("Resetting profile property and marking context dirty. Version '" + StringUtils.join(activeProfiles, ", ") + "' vs '" + StringUtils.join(profilesFromTestClass, ", ") + "'");
            testContext.markApplicationContextDirty();
            testContext.setAttribute(DependencyInjectionTestExecutionListener.REINJECT_DEPENDENCIES_ATTRIBUTE, Boolean.TRUE);
        } else {
            logger.info("Same profile as previous run detected - reusing cached version '" + StringUtils.join(activeProfiles, ", ") + "' vs '" + StringUtils.join(profilesFromTestClass, ", ") + "'");
        }

        super.prepareTestInstance(testContext);
    }
...
}

an example of one of our tests:

import dk.frj.test.Version;
import dk.frj.utils.Config;
import org.apache.log4j.Logger;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.TestExecutionListeners;
import org.springframework.test.context.support.DependencyInjectionTestExecutionListener;
import org.springframework.test.context.support.DirtiesContextTestExecutionListener;
import org.springframework.test.context.transaction.TransactionalTestExecutionListener;

import java.util.Arrays;
import java.util.Collection;
import java.util.Map;

import static org.junit.Assert.*;

/**
 *
 */
@RunWith(Parameterized.class)
@ContextConfiguration(locations = "classpath:versiontest/versionTest-config1.xml")
@TestExecutionListeners(listeners = {DependencyInjectionTestExecutionListener.class, DirtiesContextTestExecutionListener.class, TransactionalTestExecutionListener.class, ProfilesShouldChangeContextListener.class} )
public class TestUsingMultipleProfiles {
    private static final Logger logger = Logger.getLogger(TestUsingMultipleProfiles.class);

    @Rule
    public static SpringContextRule context = new SpringContextRule();

    public String[] activeProfiles;
    private int version;

    @Autowired
    private Version versionTest;


    public TestUsingMultipleProfiles(Config config, String[] springProfiles, int version) {
        this.activeProfiles = springProfiles;
        this.version = version;
    }

    @Parameterized.Parameters
    public static Collection<Object[]> profiles() {
        return Arrays.asList(new Object[][]{ //
                {Config.VERSION_1, new String[]{"profile1"}, 1},
                {Config.VERSION_2, new String[]{"profile2", "profiletest"}, 2},
                {Config.VERSION_99, new String[]{"profiletest", "profile2"}, 2},
                {Config.VERSION_3, new String[]{"profile3"}, 3}
        });
    }

    @Test
    public void versionMatchesProfile() {
        String versionString = versionTest.getTest();
        int version = Integer.parseInt(versionString.substring(versionString.length()-1));
        assertEquals("Version does not match expected profile version", this.version, version);
    }


}

The above test will be executed four times, and due to the ProfilesShouldChangeContextListener, the context will for each test run be configured with profiles mentioned in the test class. If the list of profiles is the same (no matter the order the profiles are listed in) the cached configuration is reused which can be verified using the log statements:

2012-08-20 11:40:10,361 [INFO] dk.frj.utils.ProfilesShouldChangeContextListener - Same profile as previous run detected - reusing cached version 'profile2, profiletest' vs 'profiletest, profile2'

and

2012-08-20 11:40:10,363 [INFO] dk.frj.utils.ProfilesShouldChangeContextListener - Resetting profile property and marking context dirty. Version 'profiletest, profile2' vs 'profile3'


I know that not everyone uses the @Parameterized annotation - I think a more generic solution, not requiring the use of @Parameterized runner would probably be to introduce an annotation like @ActiveProfiles but with support for a list of profilesets. And of course create a handler that would support this way of using the annotation.
I may have another go at this way of supporting multiple profile-configurations for a test in a couple of days.