Step Re-execution even when marked "isAllowStartIfComplete" as false

[vijaynairis]

Hi,

Thanks Dave, Lucas for your previous remarks.

I am still facing a problem with job re-execution.

When the jobDao and stepDao injected into simpleJobRepository is MapJobDao and MapStepDao; the re-execution starts at the step of failure.

Where as when the jobDao and stepDao injected into simpleJobRepository is SqlJobDao and SqlStepDao; the re-execution seems to start from the first step.

Please let me know what could be the cause for the same.

Thanks

[lucasward]

Are you seeing entries for the step within the meta data tables? For example, when using the Sql daos, are you seeing an entry in the BATCH_STEP_EXECUTION table indicating that the step that finished has a status of 'COMPLETED'. If not, it may be a transaction issue. If you look at the sample jobs we have, were are wrapping calls to the metadata tables in transactions via aop, are you doing the same?

[vijaynairis]

Thanks Lucas for the response,


For example, when using the Sql daos, are you seeing an entry in the BATCH_STEP_EXECUTION table indicating that the step that finished has a status of 'COMPLETED'.


The database tables BATCH_STEP_EXECUTION as well as BATCH_STEP has been updated with status 'COMPLETED' for the first 5 jobs. And for the 6th one it is updated with 'FAILED'.

The steps have been explicitly configured with 'isAllowStartIfComplete' as false.

The BATCH_JOB as well as BATCH_JOB_EXECUTION is updated with 'FAILED'.

However on a restart, a new job gets added to BATCH_JOB, the steps get
added again to BATCH_STEP, and the whole execution starts from step1.

[lucasward]

It sounds to me like you have 'restartable' on JobConfiguration set to false, which means that a step will always be recreated and started from the beginning.

[vijaynairis]

Thanks lucas.

The job configuration is configured as restartable = true. Here is a sample similar to the batch job configuration i ve done:



<bean id="batchProcessingJob" class="org.springframework.batch.core.configuration.JobCo nfiguration">
<property name="restartable" value="true" />
<property name="steps">
<list>
<bean id="step1" class="org.springframework.batch.execution.step.SimpleSte pConfiguration">
<property name="allowStartIfComplete" value="false" />
<property name="tasklet">
<bean class="com.xxx.AbcTasklet">
</bean>
</property>
</bean>
<bean id="step2" class="org.springframework.batch.execution.step.SimpleSte pConfiguration">
<property name="allowStartIfComplete" value="false" />
<property name="tasklet">
<bean class="com.xxx.DefTasklet">
</bean>
</property>
</bean>
</list>
</property>
</bean>

[lucasward]

I might have an idea what the issue is. Earlier you mentioned the following:


he database tables BATCH_STEP_EXECUTION as well as BATCH_STEP has been updated with status 'COMPLETED' for the first 5 jobs. And for the 6th one it is updated with 'FAILED'.

Can you define for me what you mean by '5 jobs.' Also, is there anyway you can do a quick dump of the meta data tables into the forums? I can understand if you have to scrub certain names, but the rest of the data should be safe.

-Lucas

[vijaynairis]

By first 5 jobs i mean the first 5 steps, configured as i ve mentioned above.

I ve attached a file with dumps of all the tables:

BATCH_JOB
BATCH_JOB_EXECUTION
BATCH_STEP
BATCH_STEP_EXECUTION

[lucasward]

The first thing I noticed when looking at your files, is that you have 36 separate steps! I would really think about breaking this up into multiple jobs and letting a scheduler launch them at the end of the day. (If cost is an issue, use Quartz) Unless each one absolutely requires the one before it to finish, you should be able to run the jobs in parallel, and finish sooner.

On to the issue itself. Looking at the BATCH_JOB table, where all of the 'Identifier's' used to identify a JobInstance are the same, it looks exactly like restartable was set to false, instead of true. Can you try 'blowing out' the tables, and running again, with restartable set to true?

[vijaynairis]

Hi Lucas,

As you would observe from the configuration, I have set restartable to "true". Blowed out the tables and ran again, same issue.

Thanks...Vijay

[lucasward]

Vijay,

As near as I can tell, the behavior I noticed in the tables you sent me is showwing that a new JobInstance is being created when you run the job again. To help illustrate, I've pulled a snippet of code form our repository:



// Check if a job is restartable, if not, create and return a new job
if (jobConfiguration.isRestartable()) {

/*
* Find all jobs matching the runtime information.
*
* Always do this if the job is restartable, then if this method is
* transactional, and the isolation level is REPEATABLE_READ or
* better, another launcher trying to start the same job in another
* thread or process will block until this transaction has finished.
*/

jobs = jobDao.findJobs(jobIdentifier);
}


So, only one of two things can be happening, either restartable is false, or findjobs is not finding the originals job instance for some reason. Can you debug through it quickly to determine which is the case?

-Lucas

[vijaynairis]

Its solved.....It was a pretty dumb mistake by me...I had commented out the job identifier factory identifier....thanks Lucas for pointing out the JobDao

<property name="jobIdentifierFactory" ref="jobRuntimeInformationFactory" />

This used the scheduled date that I completely ignored in setting while creating this instance

<bean id="jobRuntimeInformationFactory" class="org.springframework.batch.execution.runtime.Schedu ledJobIdentifierFactory">
<property name="jobKey" value="TestStream" />
<property name="scheduleDate" value="20070505" />
</bean>

So the query FIND_JOBS in the Dao would never find the job and would always execute from the beginning....sorry for the bother it was a rather stupid mistake

Also, is it possible for me to create my own scheduled job identifier factory as I would like to have the scheduled date to be picked up from a table every time and not be statically injected at once ? Any ideas ?

Thanks..VJ

[lucasward]

Yes, you can definitely create your own implementation of the JobIdentiifierFactory interface. Also, I'm working on making this even easier for milestone 4 right now (see BATCH-84)