Rollback of chunk on exception

[Dave Syer]

Configuring an ExceptionHandler for the StepOperations does the trick.

OK. I made it a bit easier (look in SVN or in tonight's snapshot) by adding the step operations to the step configuration (so you don't have to change the StepExecutor). So ChunkOperationsStepConfiguration is now called RepeatOperationsStepConfiguration and it allows you to set the step operations as well as the chunk operations.

Do you think that the ExceptionHandler on SimpleStepConfiguration should go at the step level (not the chunk level as it does now)? It seems like this is the more likely scenario for common exception handler strategies.

But I don't understand what you mean when you say that the items should be skipped.

If the exception is thrown in the ItemProcessor (or anywhere in the Tasklet if the Tasklet is Skippable and passes the message down to the ItemProvider - this is the case for the ItemProviderProcessorTasklet) the item will be marked in the InputSource as skipped and will not be presented again to the processor.

Additionally, is there a way to repeat the processing of the successfully processed items which have not been committed because of the rollback of the transaction?

That should be automatic if your ExceptionHandler in the step operations does not rethrow - the transaction will be repeated and all the unskipped items (the ones that did not fail) will be processed.

N.B. this is still not true for some failure modes using Hibernate-based ItemProcessors, until we fix or find a workaround for BATCH-194 - the item cannot be skipped because the exception came outside the Tasklet execution.

[MisterX]

Do you think that the ExceptionHandler on SimpleStepConfiguration should go at the step level (not the chunk level as it does now)? It seems like this is the more likely scenario for common exception handler strategies.

Yes, I guess this was the reason for all the confusion on this topic.

That should be automatic if your ExceptionHandler in the step operations does not rethrow - the transaction will be repeated and all the unskipped items (the ones that did not fail) will be processed.

Is there some kind of configuration to enable this behaviour? I'm using a RestartableItemProviderTasklet with a ValidatingItemProvider and I can see that the chunk is not processed again. Instead the chunk is rolled back and the processing of the next chunk begins.

[Dave Syer]

I forgot to add (maybe this is what you are finally missing) - Tasklet exceptions only cause a skip if you are using DefaultStepExecutor, and for some reason the samples are set up to use SimpleStepExecutor. I think the Default* is so little different from Simple* that I might just move that feature into Simple* and remove Default* from the API completely.

[MisterX]

I could not get it running by replacing the SimpleStepExecutor with the DefaultStepExecutor. Can you provide an example in spring-batch-samples, maybe the tradeJob.xml?

[Dave Syer]

How do you mean you "could not get it running"? If you look in SVN all the samples are using DefaultStepExecutor (as of an hour ago), but all I did was change the class name in simple-container-definition.xml. I have been using the hibernateJob.xml as a test because I am working on BATCH-194, so if you ignore the hibernate specific stuff that should work. You must be trying to do something more complicated?

[elukaweski]

Hi:

I was looking for the abstract class mentioned in the note - "added AbstractDrivingQueryInputSource superclass that encapsulates duplications from IbatisDrivingQueryInputSource and SingleKeyDrivingQueryInputSource."

I am curious as to how the framework handles iBatis when is doesn't have the capability of iterating over a result set.

Thanks

[MisterX]

Dave, I took a look at the Hibernate example and it seems to work as expected. However, when I'm using an example based on JDBC it does not show the expected behaviour. I've created a JIRA issue (Batch-244) and attached an example job to it. It would be nice if you could try to reproduce the problem and give me any hints what I'm doing wrong.

Thanks in advance.

[wxlund]

I was curious about how you're configuring Hibernate in the batch configuration. I've done a lot of Hibernate batch in the past and one of the issues we dealt with was flushing. Hibernate 3.x has added a number of new features that were not available to us but essentially we set Flush to FLUSH.NEVER and then performed the flush at each unit of work (Item).

Our mode of determining where we were at in the chunk processing was using a DrivingQueryInputSource (I'm having to translate because it was pre-Spring-batch) but essentially the DrivingQueryInputSource, because the keys are in memory, allowed us to know where we were at when the Chunk needed to re-process a Chunk. This was necessary because Hibernate requires us to throw out the Session when a transaction is rolled back. At this point you can apply a strategy to change the commit interval to commit from the start of the chunk up until the bad record and then re-adjust at that point.

Regardless, I'd love to know how your configuring the hibernate parts of batch and which, if any, Hibernate 3.x features you're using (e.g. stateless session, etc).

Wayne

[MisterX]

Wayne,

sorry for not answering sooner. Our Hibernate configuration is not any different from the one used in "normal" web applications. We do not use any of the Hibernate configuration used for batch processing. When we had our Hibernate related problems with Spring Batch we had a short look into the StatelessSession but we decided not to go with that because one of our goals is to reuse existing components from web applications in our Spring Batch based jobs.

Our current solution is to flush the Hibernate session automatically after processing each item. This is done by configuring an aspect that is invoked after the process() method of each ItemProcessor. This way we get any Hibernate related errors at least when the processing of an item (during which Hibernate is used) is finished. The Spring Batch framework is able to handle the error and can rerun the failing chunk just as it is expected.

Please have a look at BATCH-194 to see the implementation of the advice and its configuration.

Jens