Creating New MultiResourceItemReader Instance When Spring Batch Job Executes

[Jurie Oberholzer]

Hi,

I am using Spring 3.1.1 and Spring Batch 2.1.1. I want to define a Spring Batch job that will read txt files from an FTP location and process them. The FTP location is specified in a properties file 'batch.properties'. I am using a MultiResourceItemReader to process each file in turn. In the code below, the tokenizers have been omitted.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:batch="http://www.springframework.org/schema/batch"
	xmlns:context="http://www.springframework.org/schema/context"
	xmlns:util="http://www.springframework.org/schema/util" xmlns:lang="http://www.springframework.org/schema/lang"
	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:task="http://www.springframework.org/schema/task" xmlns:tx="http://www.springframework.org/schema/tx"
	xsi:schemaLocation="http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-3.0.xsd
		http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
		http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd
		http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
		http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">
		
	<bean id="placeholderPropertiesBatch"
		class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
		<property name="location" value="classpath:batch.properties" />
		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
		<property name="ignoreUnresolvablePlaceholders" value="true" />
		<property name="order" value="1" />
	</bean>	
		
	<job id="invoiceFileImportJob" xmlns="http://www.springframework.org/schema/batch" >
		<step id="invoiceFileImportStep" parent="simpleStep" next="invoiceFileRenameStep">
			<tasklet>
				<chunk reader="invoiceFileReader" writer="invoiceFileWriter" commit-interval="10" >
					<batch:streams>
						<batch:stream ref="ffItemReader" />
					</batch:streams>
				</chunk>
				<listeners>
					<listener ref="promotionListener"/>
				</listeners>
			</tasklet>
		</step>
		<step id="invoiceFileRenameStep">
			<tasklet ref="invoiceFileRenameTask" />
		</step>
	</job>
	
	<bean id="invoiceFileReader" class="za.org.joburg.shared.batch.InvoiceFileItemReader" >
		<property name="delegate" ref="ffItemReader" />
	</bean>
	
	<bean id="invoiceFileWriter" class="za.org.joburg.shared.batch.InvoiceFileWriter">
		<property name="sourceFileDAO" ref="sourceFileDAO"/>
		<property name="invoiceFileProcessor" ref="invoiceFileProcessor"/>
	</bean>
	
	<bean id="promotionListener" class="org.springframework.batch.core.listener.ExecutionContextPromotionListener" >
    	<property name="keys" value="invoiceFileList"/>
	</bean>
	
	<bean id="invoiceFileRenameTask" class="za.org.joburg.shared.batch.InvoiceFileRenameTask">
		<property name="sourceDir" value="${batch.inv.file.source}"/>
	</bean>
	
	<bean id="invoiceFileProcessor" class="za.org.joburg.shared.batch.InvoiceFileProcessor">
		<property name="taxInvoiceDocumentCreator" ref="taxInvoiceDocumentCreator"/>
		<property name="emailInvoice" ref="emailInvoice"/>
		<property name="distributionDAO" ref="distributionDAO"/>
		<property name="exceptionDAO" ref="exceptionDAO"/>
	</bean>
	
	<bean id="ffItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
		<property name="strict" value="false" />
		<property name="resources" value="${batch.inv.file.location}" />
		<property name="delegate">
			<bean class="org.springframework.batch.item.file.FlatFileItemReader" >
				<property name="strict" value="false" />
				<property name="lineMapper">
					<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" >
						<property name="lineTokenizer" ref="invoiceFileTokenizer" />
						<property name="fieldSetMapper">
							<bean class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper" />
						</property>
					</bean>
				</property>
			</bean>
		</property>
	</bean>
</beans>

This configuration file works but only loads the files that are in the source directory when the application starts up. The reason for this is because the singleton 'ffItemReader' is loaded when the application is starts up it sets the 'resources' to whatever files are in the source directory. When the job runs again it will continue looking for the same files. I understand that we want a new instance of 'ffItemReader' to be created every time this step runs in this job.

Another post on the Spring Forums (http://forum.springsource.org/showth...urceItemReader) suggested defining the scope as step on the bean. The resulting 'ffItemReader' bean can be found below,

<bean id="ffItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader" scope="step" >
	<property name="strict" value="false" />
	<property name="resources" value="${batch.inv.file.location}" />
	<property name="delegate">
		<bean class="org.springframework.batch.item.file.FlatFileItemReader" >
			<property name="strict" value="false" />
			<property name="lineMapper">
				<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" >
					<property name="lineTokenizer" ref="invoiceFileTokenizer" />
					<property name="fieldSetMapper">
						<bean class="org.springframework.batch.item.file.mapping.PassThroughFieldSetMapper" />
					</property>
				</bean>
			</property>
		</bean>
	</property>
</bean>

When I try and use the configuration above, my application fails to deploy and I get the following exceptions,

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invoiceFileImportStep': Cannot resolve reference to bean 'invoiceFileReader' while setting bean property 'itemReader'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invoiceFileReader' defined in class path resource [invoice-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.ConversionNotSupportedException: Failed to convert property value of type '$Proxy23 implementing org.springframework.batch.item.ItemReader,org.springframework.batch.item.ItemStream,java.io.Serializable,org.springframework.aop.scope.ScopedObject,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised' to required type 'org.springframework.batch.item.file.MultiResourceItemReader' for property 'delegate'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy23 implementing org.springframework.batch.item.ItemReader,org.springframework.batch.item.ItemStream,java.io.Serializable,org.springframework.aop.scope.ScopedObject,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised] to required type [org.springframework.batch.item.file.MultiResourceItemReader] for property 'delegate': no matching editors or conversion strategy found
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invoiceFileReader' defined in class path resource [invoice-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.ConversionNotSupportedException: Failed to convert property value of type '$Proxy23 implementing org.springframework.batch.item.ItemReader,org.springframework.batch.item.ItemStream,java.io.Serializable,org.springframework.aop.scope.ScopedObject,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised' to required type 'org.springframework.batch.item.file.MultiResourceItemReader' for property 'delegate'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy23 implementing org.springframework.batch.item.ItemReader,org.springframework.batch.item.ItemStream,java.io.Serializable,org.springframework.aop.scope.ScopedObject,org.springframework.aop.framework.AopInfrastructureBean,org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised] to required type [org.springframework.batch.item.file.MultiResourceItemReader] for property 'delegate': no matching editors or conversion strategy found

I understand what the problem is but I am not sure if I am using scope="step" correctly or even in the correct place.

Please could someone offer some suggestions.

Thanks and your help will be greatly appreciated!

[Marten Deinum]

You should program to interfaces and not concrete implementation. I suspect your InvoiceFileItemReader uses the MultiResourceItemReader instead of ItemReader (it should use the latter). Also your resources should probably come from t he jobParameters and not from some property place holder.

[Jurie Oberholzer]

Hi Martin,

Thanks for the advice. The InvoiceFileItemReader is implementing ItemReader but the delegate is declared as a MultiResourceItemReader. Please see an excerpt of the code below.

public class InvoiceFileItemReader implements ItemReader <SourceFile>{

	@Autowired
	private MultiResourceItemReader<FieldSet> delegate;
	
	public MultiResourceItemReader<FieldSet> getDelegate() {
		return delegate;
	}

	public void setDelegate(MultiResourceItemReader<FieldSet> delegate) {
		this.delegate = delegate;
	}

...

}

Does this look correct?

Thanks.

[Marten Deinum]

No... As mentioned you should program to the interface NOT the concrete implementation... So instead of the MultiResourceItemReader you should use the ItemReader interface.

[Jurie Oberholzer]

Hi Martin,

Thanks for your help.

I have made the changes you suggested and everything seems to be working. I now have another question with regards to how I get the filename of the resource if I am using the ItemReader interface?

Please if you could help me out.

Thanks,

[mminella]

Stupid question...if you are delegating input to a standard ItemReader implementation, why do you need access to the file name at all? A need to do so sounds like a bleeding of concerns from the delegate to the wrapper.

If you do need it, however, you can inject it into your ItemReader implementation just as you would any other property.