custom tasklet and flatfileItemWriter close error

[frankclifford]

I have a requirement to do the following:

1) Read flat files
2) Validate and transform data
3) Pass valid data to a component for further processing.

I capture invalid data using the skip listener which then puts the data into the job execution context.

The next step uses a custom tasklet to read the data, from the job execution context and write out to an invalid recs file. The flatfileItemWriter is injected into the custom tasklet

<bean id="failedItemWriter" class="com.test.writer.FailedItemWriter">
<property name="resource" value="file:target/test-outputs/CustomerReport1.txt" />
<property name="lineAggregator" >
<bean class="org.springframework.batch.item.file.transfo rm.FormatterLineAggregator">
<property name="format" value="%-9s%-3s" />
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transfo rm.BeanWrapperFieldExtractor">
<property name="names" value="name,credit" />
</bean>
</property>
</bean>
</property>
</bean>

the tasklet looks like this



@SuppressWarnings("unchecked")
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {

StepContext sc= arg1.getStepContext();
StepExecution se = sc.getStepExecution();
JobExecution je = se.getJobExecution();
ExecutionContext ec= je.getExecutionContext();

if (ec.containsKey("custList") ) {
custList = (List<CustomerCredit>) ec.get("custList");

failedItemWriter.setShouldDeleteIfExists(true);

failedItemWriter.open(ec);
failedItemWriter.write(custList);
failedItemWriter.close();


}

The problem is that I get the following exception

2009-11-01 12:21:28,911 DEBUG [org.springframework.batch.item.database.JdbcBatchI temWriter] - <Writing to flat file with 6 items.>
2009-11-01 12:21:31,669 DEBUG [org.springframework.batch.core.step.tasklet.Taskle tStep] - <Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]>
2009-11-01 12:21:33,989 ERROR [org.springframework.transaction.support.Transactio nSynchronizationUtils] - <TransactionSynchronization.afterCompletion threw exception>
org.springframework.batch.support.transaction.Flus hFailedException: Could not write to output buffer
at org.springframework.batch.support.transaction.Tran sactionAwareBufferedWriter$1.afterCompletion(Trans actionAwareBufferedWriter.java:71)
....

Caused by: java.io.IOException: Stream closed

[Jan Kronquist]

I got the same problem when migrating from Spring Batch 1.1.4 to 2.0.3. In the new version the FlatFileItemWriter is participating in the transaction and will not flush until the transaction is commited.

The problem is occurs because you manually close the FlatFileItemWriter. Removing the close() should solve the problem.

[lucasward]

I'm not sure putting it into the Job ExecutionContext is the right move. That particular ExecutionContext is only written at Step boundaries last I checked. (although there could have been a change in the last couple months in the 2.1 release, I would need to double check) Regardless, even if SB *did* update the Job EC on transaction boundaries, it seems like an extra step to write it out to the SB metadata tables, rather than just writing it out yourself. Why not have the StepListener write it out? SB puts those calls into a separate transaction, so any potential rollbacks shouldn't be an issue. (Essentially a propagation of REQUIRES_NEW)