Parallel steps issue in a job.
Hi,
i'm trying to create a simple spring batch project where i have only one JOB with 3 steps. The idea is to run these 3 steps in parallel. I have seen a lot of examples, read in various books about parallel steps in Spring Batch, however the result of the execution is pretty much not as expected, because in runtime i am monitoring the 3 created threads and they are not running concurrently. I would like to understand why and if somebody could point out to a potential solution, it would be great or maybe my XML job cofiguration is not the appropriate one.
The scenario is simple: Each step reader reads from a different file, passes through a processor (that does nothing in this case) and writes the content into a database table using JDBC.
Here is my launch-context.xml:
Code:<bean id="jobOperator" class="org.springframework.batch.core.launch.support.SimpleJobOperator" p:jobLauncher-ref="jobLauncher" p:jobExplorer-ref="jobExplorer" p:jobRepository-ref="jobRepository" p:jobRegistry-ref="jobRegistry" /> <bean id="jobExplorer" class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean" p:dataSource-ref="dataSource" /> <bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry" /> <bean class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor"> <property name="jobRegistry" ref="jobRegistry" /> </bean> <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" /> </bean> <bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean" p:dataSource-ref="dataSource" p:transactionManager-ref="transactionManager" /> <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"> <property name="driverClassName" value="${batch.jdbc.driver}" /> <property name="url" value="${batch.jdbc.url}" /> <property name="username" value="${batch.jdbc.user}" /> <property name="password" value="${batch.jdbc.password}" /> </bean> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="dataSource" /> </bean> <bean id="placeholderProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <property name="location" value="classpath:batch-default.properties" /> <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" /> <property name="ignoreUnresolvablePlaceholders" value="true" /> <property name="order" value="1" /> </bean>
and here is my job-context.xml :
Thank you, and looking forward to your answer.Code:<import resource="../override/app-context.xml"/> <bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" /> <batch:step id="file1Step"> <batch:tasklet transaction-manager="transactionManager"> <batch:chunk reader="inputReader1" processor="processor" writer="outputWriter" commit-interval="1000" /> </batch:tasklet> </batch:step> <batch:step id="file2Step"> <batch:tasklet transaction-manager="transactionManager"> <batch:chunk reader="inputReader2" processor="processor" writer="outputWriter" commit-interval="1000" /> </batch:tasklet> </batch:step> <batch:step id="file3Step"> <batch:tasklet transaction-manager="transactionManager"> <batch:chunk reader="inputReader3" processor="processor" writer="outputWriter" commit-interval="1000" /> </batch:tasklet> </batch:step> <bean id="inputReader1" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <property name="resource" value="partition0.csv" /> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="names" value="uci,companyName,companyAddrCountry,companyAddrCity,companyAddrStreet,companyAddrPostcode,creditRating" /> <property name="delimiter" value=","/> </bean> </property> <property name="fieldSetMapper"> <bean class="org.springframework.batch.admin.sample.reader.CompanySetMapper" /> </property> </bean> </property> </bean> <bean id="inputReader2" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <property name="resource" value="partition1.csv" /> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="names" value="uci,companyName,companyAddrCountry,companyAddrCity,companyAddrStreet,companyAddrPostcode,creditRating" /> <property name="delimiter" value=","/> </bean> </property> <property name="fieldSetMapper"> <bean class="org.springframework.batch.admin.sample.reader.CompanySetMapper" /> </property> </bean> </property> </bean> <bean id="inputReader3" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <property name="resource" value="partition2.csv" /> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="names" value="uci,companyName,companyAddrCountry,companyAddrCity,companyAddrStreet,companyAddrPostcode,creditRating" /> <property name="delimiter" value=","/> </bean> </property> <property name="fieldSetMapper"> <bean class="org.springframework.batch.admin.sample.reader.CompanySetMapper" /> </property> </bean> </property> </bean> <bean id="outputWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter" scope="step" > <property name="dataSource" ref="dataSource"/> <property name="sql" value="insert into company_rating (uci, company_Name, company_Addr_Country, company_Addr_City, company_Addr_Street, company_Addr_Postcode, credit_Rating) values (?, ?, ?, ?, ?, ?,?)" /> <property name="itemPreparedStatementSetter" ref="preparedStatementSetter"/> </bean> <bean id="preparedStatementSetter" class="org.springframework.batch.admin.sample.writer.CompanyPreparedStatementSetter" /> <bean id="processor" class="org.springframework.batch.admin.sample.processor.SampleItemProcessor" scope="step" /> <batch:job id="parallelJob"> <batch:split id="parallelProcessing" task-executor="taskExecutor"> <batch:flow> <batch:step id="step1" parent="file1Step" /> </batch:flow> <batch:flow> <batch:step id="step2" parent="file2Step" /> </batch:flow> <batch:flow> <batch:step id="step3" parent="file3Step" /> </batch:flow> </batch:split> </batch:job>
Reply With Quote