Disable Retry in Spring Batch not working

[Sanjeevkumar]

In my current implementation my item writer nests transactions. So problem in the call of the second method does not cause the rollback of the first method transaction.

However exception thrown by the second method results in the Spring Batch to retry and hence results in the duplicates being posted as first method in the itemwriter will also be called again.

I have set the retry-limit to zero. But it does not seem to be working. Is there any reason it is not working. I want to disable retry. Also i am skipping the records for any exception thrown.



Thanks a lot.

[Dave Syer]

Why would you want to start a new transaction in your writer?

A skip is a retry with limit=0 so you have to give up both if you want to disable the fault tolerant features entirely. The usual advice is: if you can, use an ItemProcessor for your business processing, restrict the possible exceptions from a writer to those that are truly unrecoverable, and tune the expception types.

[Sanjeevkumar]

Hi Dave,
I have an insert and a delete call insider write method

public void write()
throws Exception {
a.insert();
b.delete();
}

However problem in the delete method of b should not cause the insert() to rollback as they are isolated from each other. Hence i created a Requires_new transaction for each of these calls. Also i do not have any other business logic embedded in the write method.
Delete method currently is throwing an SQLException that is trigerring the retry i suppose.

Also i do want to skip all the records that fail without a job getting failed. Hence i am skipping all records irrespective of exception thrown.

I do have one more question. Why would a skip do a retry. Is it not possible that somebody would want to skip a record with no retry at all? Is there any special reason for a skip to have retry?

Thanks in advance.

[Dave Syer]

However problem in the delete method of b should not cause the insert() to rollback

I'm not really sure what the business use case is here. Maybe you just need to try/catch around the delete() and discard the exception then? (And keep the insert() in default REQUIRES propagation.)

I do have one more question. Why would a skip do a retry. Is it not possible that somebody would want to skip a record with no retry at all? Is there any special reason for a skip to have retry?

I guess you could see it as an implementation detail. But the semantics of retry and skip are identical, so I think it makes sense. If you want to skip with no retry, then simply set the retry limit for that exception type to be 0.

This doesn't change the fact that the framework needs to scan the chunk for the failed item in order to honour the skip listener process (to ensure that it is an item that is being skipped not a chunk). So in fact it is the skip that drives the "re-processing" that you experience.

We already made some changes here in 2.1, but I think there is room to tighten up these semantics and maybe change the implementation a bit and provide more options, so if you can come up with a really crisp description of what you need and put it in a JIRA that will help. I think there are a few new feature requests here and in related posts by Stephane:

1) option to skip/retry a whole chunk and not scan for the bad item,

2) ability to identify the failed item(s) in a skip listener,

3) ability to identify the failed item(s) that led to the skip policy being breached

[xoninhas]

Ok I don't wanna be a digger here but... this has been open 2 years ago and there is still no elegant way to just skip exceptions?

Don't get me wrong again but, do you really call scanning executing the processor and the writer again? As far as I know if I fail at something and I go do it again I call that retry... I didn't go scan the math exam, I retried it. I'm sorry I'm pushing it with this but I think the concept of scanning was totally missed.

As far as I know I have no workaround that would prevent spring batch from retrying the operation two years after this issue was posted.

The issue that is connected with this thread has a minor priority which I also don't understand since it seems I have to code on the processor/writer which items I processed or written so that I can skip them, meaning the skipLimit is not working! SkipLimit is intended to not care which item failed otherwise I would use a skipPolicy.

Why couldn't the writer/processor have some context where they could mark an item that threw exception and that would be it, then checking that map/list could actually be called scan.

I'm available to help out with coding something with some pull request, if someone takes the time to run me through some of the code and concepts. Since all I need is that "commit-interval=1" with "skipLimit=100" works as written, with no retrying, I'm probably not seeing all the scenarios.

I explained to a colleague this problem now and his question was why does spring batch want to know which item failed? Did you ask him to do that?... Well my answer is no I didn't ask SB to do that, in fact I don't care which item failed cause I log which item failed, like a responsible programmer .

Can someone explain me why is this so called "scanning" even useful for? For retry policies (I don't have one)? For skip policies (I have just a skipLimit)? What does SB do with the item that it found to have failed?