Noob: Why would Spring Batch kick off three threads for one job?

[desropolis]

I'm very new to Spring Batch, but I see enormous value in the tool. Where I work we're moving off COBOL and on to Java, so such a framework has great potential here.

For some reason my Spring Batch configuration starts three concurrent threads for my job. Since I use the JpaPagingItemReader this is a major problem. Several threads are all trying to open and close the same transaction.

I made a thread-safe version of JpaPagingItemReader just for testing and all three threads executed perfectly in series. I could just use this (extremely naive) thread-safe version, but I'd like to know what could cause Batch to kick off three threads without being explicitly configured to do so.

Thanks,
Dean

[Dave Syer]

Spring Batch doesn't spin threads on its own, you have to ask it to do that. Can we see your configuration?

[desropolis]

Here it is. Thanks for getting back t me so quickly. We had a holiday here in the States on Monday.

xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
	xmlns:p="http://www.springframework.org/schema/p" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:jee="http://www.springframework.org/schema/jee"
	xmlns:context="http://www.springframework.org/schema/context"
	xsi:schemaLocation="
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
    http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd
    http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-2.5.xsd
    http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd">

	<!-- Enables Spring Annotation scanning -->
	<context:component-scan base-package="org.trs" />

	<tx:annotation-driven transaction-manager="transactionManager" proxy-target-class="true" />

	<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
		destroy-method="close" p:driverClassName="com.ibm.as400.access.AS400JDBCDriver"
		p:url="jdbc:as400://system2;user=WASVALDB;libraries=UPSVALDDL,*LIBL;password=server1;meserver=qdev400;translate binary=true;date format=iso;naming=system" />

	<bean id="entityManagerFactory"
		class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
		<property name="jpaVendorAdapter">
			<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"/>
		</property>
		<property name="jpaPropertyMap">
			<map>
			    <entry key="hibernate.show_sql" value="true" />
			    <entry key="hibernate.use_sql_comments" value="true" />
			    <entry key="hibernate.format_sql" value="true" />
			    <entry key="hibernate.dialect" value="org.hibernate.dialect.DB2400Dialect" />
			    <entry key="hibernate.archive.autodetection" value="class, hbm"/>
			</map>
		</property>
		<property name="dataSource" ref="dataSource" />
	</bean>

	<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
		<property name="entityManagerFactory" ref="entityManagerFactory" />
	</bean>

	<bean id="ps_unprocessed" class="org.trs.entity.enums.ProcessingStatus"
		factory-method="valueOf">
		<constructor-arg>
			<value>PS_UNPROCESSED</value>
		</constructor-arg>
	</bean>

	<bean id="itemReader"
		class="org.springframework.batch.item.database.JpaPagingItemReader">
		<property name="entityManagerFactory" ref="entityManagerFactory" />
		<property name="pageSize" value="1000"/>
		<property name="queryString"
			value="from FormRequest fr where fr.requestProcessingStatus=:status" />
		<property name="parameterValues">
			<map>
				<entry key="status" value-ref="ps_unprocessed" />
			</map>
		</property>
	</bean>

	<bean id="outputResource" class="org.springframework.core.io.FileSystemResource">
		<constructor-arg value="target/test-outputs/delimitedOutput.csv" />
	</bean>

	<bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
		<property name="resource" ref="outputResource" />
		<property name="lineAggregator">
			<bean class="org.trs.service.ivr.business.FormRequestLineAggregator" />
		</property>
	</bean>

	<batch:job id="formRequestMailingLabelJob">
		<batch:step id="writeFormRequests">
			<batch:tasklet>
				<batch:chunk task-executor="taskExecutor"
					reader="itemReader"
					writer="itemWriter"
					commit-interval="1000"/>
			</batch:tasklet>
		</batch:step>
	</batch:job>

	<bean id="taskExecutor"
		class="org.springframework.core.task.SimpleAsyncTaskExecutor"/>

	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="isolationLevelForCreate" value="ISOLATION_DEFAULT"/>
		<property name="transactionManager" ref="transactionManager"/>
	</bean>

	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="taskExecutor" ref="taskExecutor" />
		<property name="jobRepository" ref="jobRepository" />
	</bean>

	<bean id="jobRegistry"
		class="org.springframework.batch.core.configuration.support.MapJobRegistry" />

	<bean id="jobRegistryBeanPostProcessor"
		class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
		<property name="jobRegistry" ref="jobRegistry" />
	</bean>

</beans>

[lucasward]

Considering you have an async task executor on your JobLauncher, and are using the MapJobRepository, I would say the most likely scenario is that you launched it three times, and since your reader isn't step scoped, each instance of the job is using the same instance. Step scoping would create separate instances of the reader, but you'll end up with duplicated data. I would consider only launching the job once. If it's not performing, you could then think about threading within the step, or partitioning.

[Dave Syer]

There is also a task executor in the step there, which means even in a single execution you will see multiple threads (and they probably won't be doing what you want then to unless the readers and writers are stateless).

[desropolis]

I figured out that I was, in fact, launching the job three times due to various configurations. I have it working now. Thaks for your time.