Configuring multiple jobs, CLI launching, and tests

[jeffjensen]

What is the recommended approach(es) to arrange the Spring Batch context files for multiple jobs and their shared beans? And how do you launch them from CLI?

Launching a job from CLI can only take one context file, so in our current setup I made a "spring-batch.xml" file to hold the common Spring beans for all batch jobs and a Spring context file per batch job that imports the common one. To run from CLI, I specify the specific batch job context file and its job name. What are any better approaches to suggest?

For testing the jobs, with our current setup, only one job is loaded per Spring Test context (by specifying @ContextConfiguration location) and it is magically run:

jobExecution = jobLauncherTestUtils.launchJob(jobParameters);

Note that jobParameters only has the runAsDate.

This all worked nicely, except now I've had to add JTA for the transactions. The problem encountered is each test launches another JTA config, and the JTA manager throws exception as it is already loaded (from prior test).

To solve, a colleague suggested I convert to using the AutomaticJobRegistrar so our batch tests can define only one context configuration for all jobs, thereby preventing the additional JTA manager problem.

In starting to change to that approach, it appears I should/can:
- for CLI run, specify the common context file instead of the specific batch job file (all job files will be loaded via the registrar, so no need to specify the specific file on the CLI run)
- then can avoid having each job's Spring context import the common one (as it will be loaded by the job runner for CLI and by @ContextConfiguration for tests; getting rid of imports is a good thing (TM))


When using the AutomaticJobRegistrar, what is the recommended practice(s) to run the correct job with tests? Initial analysis suggests:
- inject MapJobRegistry to test class
- call

registry.getJob("theJobNameUnderTest")

- call

jobLauncherTestUtils.setJob(theJob)

- call

jobLauncherTestUtils.launchJob(jobParameters)

as usual

Are the better recommended approaches for configuration design and tests?

I am also wondering if I need to use JobRegistryBackgroundJobRunner instead of SimpleJobLauncher when moving to the AutomaticJobRegistrar?

[lucasward]

You're kicking this off from CLI, but it's interacting with a JTA manager? I'm confused if this is in a managed Java EE environment with a CLI hook, or actual standalone J2SE JVMs?

Personally, I prefer to stick to J2SE if at all possible, because I like to have a JVM per Job. This gives me a lot more control as to individual job JVM parameters. (such as heap, GC settings, etc)

My first inclination when someone says they need JTA is 'really?' JTA is a pretty massive overhead, and even with the best implementations is impossible to guarantee that 100% of transactions will be correct. There are lots of alternatives that can be even more reliable when coded by someone that knows what the data means. However, there are situations that require it, and I've personally used some standalone JTA implementations that have worked just fine for me, even with multiple batch jobs. I would need to hear more about your specific environment before offering any solutions.

[jeffjensen]

Hi, thank you for the reply.

Sorry I wasn't clear - the questions pertain to Spring Batch in standalone JSE. I plan to have each job launched individually/in their own JVM.

Our system apps are batches, a web app (in Tomcat), and a (yet to be written but probably Spring Rich Client) remote/often disconnected app.

The JTA need came from having 4 data sources with each app using at least 2 of them, sometimes writing to multiple in one transaction.

We have 4 batch jobs, 3 with one step and 1 with 4 steps, each configured to launch separately. The real problem we are working around is not loading the standalone JTA manager multiple times across the tests. After some rearranging and establishing a top-level @ContextConfiguration for all tests that load the persistence beans, some of the tests work again; refactoring is incomplete though, and continues to evolve as we learn how we can configure/design this.

Through this work, questions of proper context file design for the jobs, tests, etc. arise. I'm not sure yet if AutomaticJobRegistrar is needed or not (currently I am hoping not; the restructuring may allow the jobs to remain separately loaded). The real key was establishing a top level @ContextConfiguration (just happened yesterday). I haven't found docs explaining, but it appears to be isolated contexts/parent-child.