File Reading twice in UNIX

[sundarvarad]

Hi all,
we are using spring batch 1.1.4 for our batch needs. In our spring batch application, we have around 4 files say

xyz_f1
xyz_F2
xyz_f3
xyz_f4

we are using MultiResourceItemReader to read multiple files. spring batch reads all 4 files fine in Windows environment. However when we tried to execute the same spring batch program in UNIX environment, it skips the file xyz_f4 and reads the file xyz_f1 twice. This looks weird to us.

Did anyone come across this? We are puzzled on how it reads the files twice in UNIX and not in Windows.


Appreciate your help

Thanks,
Sundar

[Dave Syer]

We do all our CI builds on Linux so I doubt if it is a platform issue. Can you provide a test case or some more details of what you did?

[sundarvarad]

Hi,
We are trying to run this batch in AIX UNIX environment. Here is the xml snippet we use for reading files.

<bean id="rateactionstep-BCC_CA" class="org.springframework.batch.core.step.item.SkipLimitStepFactoryBean">
		<property name="transactionManager" ref="tm"/>
		<property name="jobRepository" ref="jobRepository"/>
		<property name="commitInterval" value="10"/>
		<property name="skippableExceptionClasses" value="com.wellpoint.eas.rateaction.validator.SkippableValidationException"/>
		<property name="fatalExceptionClasses" value="com.wellpoint.eas.rateaction.validator.NonSkippableValidationException"/>
		<property name="skipLimit" value="10"/>				<property name="streams" ref="fileItemReader" />
		<property name="itemReader" ref="validatingItemReader"/>
		<property name="itemWriter" ref="transformingWriter_BCC_CA"/>
		<property name="listeners" ref="stepListener" />
	</bean>
	<bean id="fileItemReader" parent="multiResourceReader"
		autowire-candidate="false" />
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<!-- 
		<property name="resource" value="classpath:BCC_CA_032709_Monthly_F1.txt" />
 -->
		<property name="lineTokenizer" ref="RateActionTokenizer"/>
		<property name="fieldSetMapper" ref="rateActionFieldsetMapper"/>
	</bean>

	<bean id="validatingItemReader" class="org.springframework.batch.item.validator.ValidatingItemReader">
		<property name="itemReader" ref="multiResourceReader"/> 
		
		<property name="validator" ref="itemValidator"/>
	</bean> 

	<bean id="itemValidator" class="com.wellpoint.eas.rateaction.validator.RateActionValidator"/>
	<bean id="jobListener" class="com.wellpoint.eas.rateaction.listeners.JobListener">
	<property name ="emailService" ref="emailService" />
<property name ="rateActionAnniv" ref="rateActionAnniv"/>
	<property name ="rateActionService" ref="rateActionService"/>
	</bean>
<bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiResourceItemReader"> 
<property name="resources" value="file:BCC_CA_*.txt"/>
<property name="delegate" ref="flatFileItemReader"/>
</bean>

we have placed our four files in the our unix directory where we have our Jar files.

BCC_CA_041009_M_070109_F2.txt
BCC_CA_041009_M_070109_F5.txt
BCC_CA_041009_M_070109_F6.txt
BCC_CA_041009_M_070109_F1.txt

The job reader F1, F2, F5 files and instead of reading F6 files, it again reads BCC_CA_041009_M_070109_F1.txt twice.

I hope this helps you.

Thanks,
Sundar

[Dave Syer]

That looks pretty weird to me: the flatFileItemReader has a parent="multiResourceItemReader", which in turn has a delegate ref="flatFileItemReader". It's circular, so I'm not sure how to predict what should happen. I'm surprised it works at all. What happens if you ditch the parent="..." in the flatFileItemReader?

[sundarvarad]

Hi Dave,
we figured out where the problem was

In the xml,

<bean id="validatingItemReader" class="org.springframework.batch.item.validator.ValidatingItemReader">
		<property name="itemReader" ref="multiResourceReader"/> 
		
		<property name="validator" ref="itemValidator"/>
	</bean>

It should have been

		<property name="itemReader" ref="flatfileItemReader"/>

This worked fine both in Windows and UNIX environment. Yes, it was weird how it worked in windows.

Thanks for your input