Writer, Processor called twice on Exception.

[nagolusidhartha]

i have been using spring batch for a while, but still have a very basic question regarding skip. i am using the spring-batch-simple-cli project provided by spring samples, and trying to understand the skip behaviour. this has a very basic example reader that reads from an array of strings ( i have modifed it to read from a list of 10 strings starting from Hellowworld 1 to Hellowworld 10) and a basic writer that logs to console. writer throws java.lang.Exception on each write. i have added a skip limit of 4 to job configuration. once reader, writer it reached Hellowworld 5, the job stops as expected. But whenever writer throws exception, the writer is called back immediately with the same item.
that is writer is called twice for Hellowworld 1 to Hellow 5. resulting in 9 write calls.
My question why is writer called twice? i am expecting this item to be just skipped? is there something i am missing. i noticed that even if i add a processor to the below job, both processor and writer are called on skip. i have verified this bahaviour in Spring 2.0.3 and spring 2.1.8. how can i make sure that an item is skipped on s=exception, not trying twice? my job config is added below. i have put a debugger in read and write and verified the behaviour.
The spring version i am using is spring 3.0.5.
<job id="job1" xmlns="http://www.springframework.org/schema/batch" incrementer="jobParametersIncrementer">
<step id="step1" parent="simpleStep">
<tasklet>
<chunk reader="reader" writer="writer" skip-limit="4" >
<skippable-exception-classes>
<include class="java.lang.Exception" />
</skippable-exception-classes>
</chunk>
</tasklet>
</step>
</job>

[snicoll]

Look at the way a chunk is processed. Say you have a commit interval of 5, what would happen is that your reader would be called five times (reading the first 5 items), your processor would be called 5 times and your writer will be called ONE time (with the 5 items that result from the processing.

If an error is thrown in the writer, there's no way for Spring batch to know which item has failed so what it does is retrying the items one by one until it finds the culprit one (to skip it if that's what the policy states).

HTH

[xoninhas]

If an error is thrown in the writer, there's no way for Spring batch to know which item has failed so what it does is retrying the items one by one until it finds the culprit one (to skip it if that's what the policy states).

HTH

I guess that case is understandable although the problem is that it does the same even if the commit-interval is one and that is not understandable.

I think this part of the spring batch was overly complicated and actually now doesn't give so much freedom to the programmer. If I have a skip-limit, what I'm interested in is the number of exceptions that are thrown, I don't need to know which item failed. If I need to know which one, then it would actually be better if the writer could have some context and add the items that failed, instead of spring going through them allover again...


Can you maybe tell me where I can find that part of the code in spring batch that calls the writer one-by-one?

[snicoll]

I guess that case is understandable although the problem is that it does the same even if the commit-interval is one and that is not understandable.

You may want to look at: https://jira.springsource.org/browse/BATCH-1518