ItemReader commit not happenning

[patb23@gmail.com]

I am experiencing a strange problem with Item Reader/Writer with Async Task Executor and SimpleStepFactoryBean.
I tweaked the parallelJob (example) reader/writer implementation to suit my needs.
I am able to run the job successfully only when I run in the debug mode. Also, this job is working fine when I deployed in a servlet container.

I have set the 'throttleLimit' property a non-default value of 20 and when I removed that, the job worked fine.

Is it a bug or because of incorrect usage?

Thanks

[Dave Syer]

Can you provide a simple configuration and unit test that shows the issue? It could be some sort of crossed wire between the throttle limit and the task executor settings. Or your "tweaks" might have been introducing extra state that isn't thread safe?

[patb23@gmail.com]

Yes the 'throttleLimit' is not working only with my code.

My 'tweaks' are limited to changing only the SQL query and am attaching the file. In the writer, I am making a webservice call.

These are the things I noticed while testing:
1. I believe that task executor creates as many 'reader' thread as throttleLimit. All the threads that read the object from the DB are completing the writer method call.
2. throttleLimit greater than 8 is failing.
3. The threads are hanging even If I do nothing in the writer (I just placed logger statements in the write). Hence I guess writer is not introducing the problem.

Please let me know, if you need any other details in helping me solve this.

Thanks

	<bean id="simpleStep" class="org.springframework.batch.core.step.item.SimpleStepFactoryBean"
		abstract="true">
		<property name="transactionManager" ref="transactionManager" />
		<property name="jobRepository" ref="mapJobRepository" />
		<property name="startLimit" value="100" />
		<property name="commitInterval" value="1" />
	</bean>


				<bean id="loading" parent="simpleStep">
					<property name="throttleLimit" value="8"/>
					<property name="taskExecutor">
                                         .....................

[Dave Syer]

I don't think you actually said what the problem is. How does it fail? If you can provide a unit test that would be perfect.

[patb23@gmail.com]

As you said, It seems that cross-wired settings between SimpleStep and the TaskExecutor is the reason.

I ran the 'ParallelJobFunctionTests' that came with the Spring Sample and I added only the throttleLimit property. I could see the same behavior.

Anyways that I could specify the throttleLimit here?

				<bean id="loading" parent="simpleStep">
					<property name="throttleLimit" value="9"/>
					<property name="taskExecutor">
						<bean
							class="org.springframework.core.task.SimpleAsyncTaskExecutor">
						</bean>	
					</property>
					<property name="itemReader">
						<bean
							class="org.springframework.batch.sample.item.reader.StagingItemReader">
							<property name="lobHandler"
								ref="lobHandler" />
							<property name="dataSource"
								ref="dataSource" />
						</bean>
					</property>
					<property name="itemWriter">
						<bean
							class="org.springframework.batch.sample.item.writer.TradeWriter">
							<property name="dao" ref="tradeDao" />
						</bean>
					</property>
				</bean>