Spawjning multipe jobs from within a Job

[spongybob]

Hi,
I have the need to start several jobs asynchronously after processing some input.
All this is to be done from the ItemProcessor of a Job step.
If I attempt to do the above, I get an error to the tune of: Transaction already started... set the @Transactional annotation on the client..

Looking at the namespace config, it seems that it is not possible to start a Job from within another Job. It is possible to start multiple steps, tho'

However, I want to start an indeterminate number of jobs because I would like to monitor them using JobExplorer and manage their starting/stopping using JobOperator. These niceties are not available to me if I use Steps.

Can someone suggest another way to spawn "jobs" programitacally that can then be monitored using the JobOperator interface?


Thanks!

[Dave Syer]

The only limitation is that you can't launch a Job from inside a transaction. Note there is a JobStep, so it is definitely possible to start a Job from within a Job. You could use that in conjunction with a partition Step to create a variable number of Jobs at runtime based on the result of a partitioner. that would probably be the most elegant solution, depending on your requirements. Or you could control the transaction boundaries in your existing setup, e.g. by starting a new transaction, or switching off the existing one (every tasklet step has a configurable tx propagation setting).

[spongybob]

Dave,
Thanks for the reply and the suggestion to use a JobStep.
However, the way I am determining the granularity of my jobs and steps is that restartable units are deemed as 'jobs' because those are what are exposed by the JobOperator class, not the JobSteps.

So given that I have to be able to restart/stop these units, the only option I have is to spawn jobs, not steps.

As you pointed out there is a setTransactionalAttribute() method on TaskletStep. But that will not help as I was looking for a similar API on the Job interface instead.

Here is the code that I am trying and failing with the "Transaction already started... set the @Transactional annotation on the client.." issue.

	
public class SplitFilesTasklet implements Tasklet {
	
	//jobRepository and jobLauncher are injected
	
	//Tried setting TransactionalAttribute as below but that doesn't help
	//@Transactional(propagation = Propagation.SUPPORTS)
	
	
	public RepeatStatus execute(StepContribution contribution, 
                                ChunkContext chunkContext) throws Exception {
		
		Map<String, JobParameter> parameters = new HashMap<String, JobParameter>();
				parameters.put("timestamp", new JobParameter(new Date().getTime()));
		
		SimpleJob job = null;
		
		for (int i = 0; i < 5; i++) {
			job = new SimpleJob("test-" + i);
			TaskletStep taskletStep = new TaskletStep(); 
			taskletStep.setName("step");
			taskletStep.setJobRepository(jobRepository);
			taskletStep.setTransactionManager(transactionManager);
			Tasklet tasklet = new SingleFileTasklet();
			taskletStep.setTasklet(tasklet);

			job.addStep(taskletStep);
			job.setJobRepository(jobRepository);

			JobExecution jobExecution = jobLauncher.run(job,parameters);
		}

	} 	
}

The above code is in a class called SplitFilesTasklet that is itself configured to be launched using the SimpleAsynchExector and a QuartzScheduler.

When the jobLauncher attempts to launch the job above, it issues the exception.

Thanks!

[Dave Syer]

I think maybe you will have an easier time if you align your domain with the Batch one a little because I can't see anything terribly unusual in your requirements (but please explain more clearly if you disagree). A JobStep is just as restartable as the Job it contains, and I would guess that its executions can even be individually controlled independent of the parent Job. The reason for having a parent Job would be so you get control and management information of the whole group. If you don't need that then just launch them as standalone jobs.

[sky33]

This is another area that I have used Spring Integration in combination with Spring Batch. If I need to trigger a another job execution from anywhere in a running job, I just dump a message to a message channel with the job parameters as header values. With integration, now you have full flexibility with what you want to do with the message. What I will typically do then is use integration to send a JMS message with the job request data to a queue. Then you can use integration to receive the inbound message and write a very simple JMS Job Launch Request handler to get the job parameters from the message and launch the job. The beauty of using JMS in this case is that the consumer of the message can be in a different process or even on a different machine.

[Martin1973]

Dave,
Do you have an example of a partitioner launching jobsteps? My problem is that I got an JobInstanceAlreadyCompleteException when the partitioner launches the second jobstep instance.