Jobs within a Job

[spongybob]

Hi,
I would like to programmatically start some jobs from within a Job that has been launched asynchronously.

When I do so, I get the following:

java.lang.IllegalStateException: Existing transaction detected in JobRepository.Please fix this and try again (e.g. remove @Transactional annotations from client).

This is obviously happening because the parent job is not complete, so the transaction is still "active". How can I force the child jobs to join that transaction instead of complaining.

I have written a Tasklet from where I am launching these jobs. I tried setting the @Transaction attribute on the execute(...) method to SUPPORTS explicitly, but that did not help.

Can someone please suggest how I can achieve the starting of several jobs from within a job and not run into this transaction issue.

The full stack is:

2011-07-21 14:16:31,714 [SimpleAsyncTaskExecutor-18] ERROR org.springframework.batch.core.step.AbstractStep  #### Encountered an error executing the stepjava.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove @Transactional annotations from clie
nt).
        at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:164)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
        at $Proxy2.createJobExecution(Unknown Source)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:111)at com.xxx.adc.batch.SplitFilesTasklet.execute(SplitFilesTasklet.java:92)
        at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:386)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:264)
        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:214)
        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
        at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:250)
        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
        at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:135)
        at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
        at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run
(SimpleJobLauncher.java:120)

Thanks!

[howieB]

<bean id="jobRepository"
class="org.springframework.batch.core.repository.s upport.MapJobRepositoryFactoryBean">
<property name="validateTransactionState" value="false"/>
</bean>

This should do the trick... I an not sure if it's safe to do that. More on that:

https://jira.springsource.org/browse/BATCH-1668

[fusionobject]

Any other method of doing this ?

i have a reader , which read data-base, gets data, now for each record, i need to query the different database to get a lot of data and write to DB.
i can use the Item processor, but the data is huge, i cannot hold the data in memory and pass it to writer.

How do i process this ?

[Dave Syer]

If you really want to launch a Job from within a Job why not use a JobStep (it has XML namespace support as well)?

[fusionobject]

Hi Dave,
Nice idea,
For every record in the reader, i need to call a job, and pass that row value as parameter ?

can you just post me a simple example ?

[spongybob]

This seems to work:

	<job id="topJob" xmlns="http://www.springframework.org/schema/batch"  >
		<step id="topJob.step1" next="topJob.step2">
			<job ref="decryptJobStep"  job-parameters-extractor="jobParametersExtractor"/>
		</step>
		<step id="topJob.step2" next="topJob.split3">
		    <job ref="processXXXJobStep" job-parameters-extractor="jobParametersExtractor"/>
		</step>
						    
		<split id="topJob.split3" task-executor="myTaskExecutor" >
		    <flow>
		        <step id="topJob.step3" >
		        	<job ref="processYYY1JobStep" job-parameters-extractor="jobParametersExtractor"/>
		        </step>
		    </flow>
		    <flow>
		        <step id="topJob.step4" >
		        	<job ref="processYYY2JobStep" job-parameters-extractor="jobParametersExtractor"/>
		        </step>
		    </flow>
		    <flow>
		        <step id="topJob.step5" >
		        	<job ref="processYYY3JobStep" job-parameters-extractor="jobParametersExtractor"/>
		        </step>
		    </flow>
		    <flow>
		        <step id="topJob.step6" >
		        	<job ref="processYYY4JobStep" job-parameters-extractor="jobParametersExtractor"/>
		        </step>
		    </flow>
		</split>
	</job>
	


	<job id="decryptJobStep" xmlns="http://www.springframework.org/schema/batch"  >
		<step id="decryptJobStep.step1">
		    <tasklet ref="decryptTasklet" />
		</step>
	</job>
	<bean id="decryptTasklet" class = "com.acme...DecryptFileTasklet" >
	   <property name="jobRepository" ref="jobRepository" />
	   <property name="jobLauncher" ref="jobLauncher" />	
	</bean>
	
	<job id="processYYY1JobStep" xmlns="http://www.springframework.org/schema/batch"  >
		<step id="processYYY1JobStep.step1">
		    <tasklet ref="processYYY1Tasklet" />
		</step>
	</job>
	<bean id="processYYY1Tasklet" class = "com.acme...ProcessYYY1Tasklet" >
	   <property name="jobRepository" ref="jobRepository" />
	   <property name="jobLauncher" ref="jobLauncher" />	
	</bean>

HTH

[fusionobject]

Hi spongybob,

For every record in the reader, i need to call a job, and pass that row value as parameter ?
can you just post me a simple example ?

Your example, looks like, sequential job execution

[spongybob]

Hi,
I don't have an example but isn't this what you want:

<job id="topJob" xmlns="http://www.springframework.org/schema/batch"  >
		<step id="topJob.step1">
			<job ref="aJobStep" />
		</step>

</job>

<job id="aJobStep" xmlns="http://www.springframework.org/schema/batch"  >
	<step id="aJobStep.step1">
	    <tasklet>
	    	 <chunk reader="itemReader" writer="itemWriter" processor="itemProcessor"/>
	    </tasklet>
	</step>
</job>

[fusionobject]

This topJob will run only once.

But what i wanted to do is;
Job-A read DB and fetch N rows, for each ROW run JOB-b

[Dave Syer]

I think I would recommend an alternative approach where you use a Partitioner instead of an ItemReader to split up the work. If you really want to do it with an ItemReader you need to launch your sub-jobs without a transaction (as you already found) - there is probably more than one way to do that, but the easiest would be to switch off the transaction in the TaskletStep using a custom propagation.