Delegate pattern with multiple BatchSqlUpdateItemWriter cleans resources prematurely

[jmcnulty25]

I'm taking over code of another developer that recently left so please excuse my ignorance as this is my first usage of Spring Batch (final 1.0.0 version).

Below you can note that I have created a custom item writer that delegates to multiple instances of BatchSqlUpdateItemWriter.

In the flush method of my BatchSqlBillingInformationWriter, an implementation of ItemWriter, I call the flush method of each of the delegates. The problem is that in the first call to the delegate.flush() ends up calling BatchSqlUpdateItemWriter.doFlush(), in this method the finally clause calls getProcessed().clear().

This method in effect, removes the process data subsequent delegate.flush() calls would use (since they are all instances of BatchSqlUpdateItemWriter as well and the process data is bound to TransactionSynchronizationManager under the static key ITEMS_PROCESSED (in the bindTransactionResources), instead of each instance having it's own process data. This means that those delegates end up not doing any work at all as the doFlush method can not find any during look up.

Should I be doing this a different way? Let me know if more information is required.

Your help is greatly appreciated!
-Jassen

<bean id="itemWriter" class="mine.BatchSqlBillingInformationWriter">
		<property name="delegate">
			<bean class="org.springframework.batch.item.database.BatchSqlUpdateItemWriter">
				<property name="jdbcTemplate" ref="jdbcTemplate" />
				<property name="sql"><value><![CDATA[xxx]]></value></property>
				<property name="itemPreparedStatementSetter">
					<bean class="com.x.BillingProfileInsertPreparedStatementSetter">
                        <property name="sequenceIncrementer">
                            <bean class="org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer">
                                <constructor-arg><ref bean="dataSource"/></constructor-arg>
                                <constructor-arg value="xxx"/>
                            </bean>
                        </property>
                    </bean>
				</property>
			</bean>
		</property>
        <property name="updaterDelegate">
			<bean class="org.springframework.batch.item.database.BatchSqlUpdateItemWriter">
				<property name="jdbcTemplate" ref="jdbcTemplate" />
				<property name="sql"><value><![CDATA[UPDATE xxx]]></value></property>
				<property name="itemPreparedStatementSetter">
					<bean class="x.UserInformationUpdatePreparedStatementSetter">
                </bean>
               </property>
           </bean>
        </property>
        <property name="billingInstrumentDelegate">
			<bean class="org.springframework.batch.item.database.BatchSqlUpdateItemWriter">
				<property name="jdbcTemplate" ref="jdbcTemplate" />
				<property name="sql"><value><![CDATA[INSERT INTO xxx]]></value></property>
				<property name="itemPreparedStatementSetter">
					<bean class="com.xxx.BillingInstrumentInsertPreparedStatementSetter">
                        <property name="sequenceIncrementer">
                            <bean class="org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer">
                                <constructor-arg><ref bean="dataSource"/></constructor-arg>
                                <constructor-arg value="xxx"/>
                            </bean>
                        </property>
                    </bean>
				</property>
			</bean>
		</property>
        <property name="insertDelegate">
			<bean class="org.springframework.batch.item.database.BatchSqlUpdateItemWriter">
				<property name="jdbcTemplate" ref="jdbcTemplate" />
				<property name="sql"><value><![CDATA[INSERT INTO xxx]]></value></property>
				<property name="itemPreparedStatementSetter">
					<bean class="xxx.SourceInsertPreparedStatementSetter"/>
				</property>
			</bean>
		</property>
        <property name="updaterDelegate">
			<bean class="org.springframework.batch.item.database.BatchSqlUpdateItemWriter">
				<property name="jdbcTemplate" ref="jdbcTemplate" />
				<property name="sql"><value><![CDATA[UPDATE xxx]]></value></property>
				<property name="itemPreparedStatementSetter">
					<bean class="xxx.UpdatePreparedStatementSetter">
                </bean>
               </property>
           </bean>
        </property>
	</bean>

[lucasward]

The BatchSqlUpdateItemWriter will definitely have issues if you have more than one in the current step. The problem is that it has to do a lot of work to keep track of failed records, since executing all statements at once would prevent you from knowing which item caused a failure. I suppose we could provide a way to set the key that's used to store the items, but it seems a bit hackish.

I'm curious if you really need a batch update writer? I'm personally not the biggest fan of it. You would really only see a performance benefit if you have a large commit interval with extremely clean data. I would switch them over to be normal jdbc inserts (just create a custom dao, or use SimpleJdbcInsert if you have access to spring 2.5) and if it performs fast enough for you, call it a day. If not, there's probably some work-arounds you could try to solve the issue.