Skipping is not working for multistep with multi threading
Hi ,
How impliment skipping for multi threading
<bean id="copyAll1" parent="multiStep">
<property name="itemReader">
<bean class="caf.opm.cpm.feed.common.spring.batch.
MultiItemReader" p:lazy="true">
<property name="readers">
<list>
<ref bean="forecastEXPReader" />
<ref bean="pdbRWAReader" />
<ref bean="pharosRWAProfitCenterReader" />
<ref bean="pharosRWASubBusinessUnitReader" />
<ref bean="pharosRWABusinessUnitReader" />
<ref bean="pharosRWAFacilitiesPortfolioReader" />
<ref bean="pharosRWADealsPortfolioReader" />
</list>
</property>
</bean>
</property>
</bean>
For these step parent is configured as “multiStep”, which uses sharedThreadPoolTaskExecutor for step execution. So it is not possible to handle the skipping mechanism in as we are using multi threading for writing items into DB, write skips do not work in a multi-threaded step.
<bean id="multiStep" parent="skippingStep" abstract="true">
<property name="itemWriter" ref="multiWriter" />
<property name="throttleLimit" value="100" />
<property name="taskExecutor"
ref="sharedThreadPoolTaskExecutor" />
<property name="listeners" ref="stepListeners" />
<property name="commitInterval" value="100" />
</bean>
if i have exception in first record then job is crashing not proceeding for next record.
Expected : We would like to avoid job crash when input data contains bad format lines. When we get an exception or error while persisting data in ERROR_TABLE and the job should feed the good formatted lines and persist in target tables. Only bad format lines should be skipped and logged into functional error table till it reaches the skip limit.
Reply With Quote
I agree with you
