Writing to Multiple file

[Jainendra]

Hi,
My requirment is. I have to read records from a single file. Based on the some business logic I have to write this record to either of the three file.
1. BCPFile (Valid record)
2. NewRecordFile (This file will have the new record which doesnot match with the existing record in database)
3. InvalidRecordFile (File that has invalid record).

Can I use composite writer here. Or do i need to use some transformer. I tried using composite writer but it writes twice in single file, instead of writing on different file

In spring batch do we have something that deals with the above scenario

Its very urjent.

[robert.kasanicky]

The CompositeItemWriter provided by the framework simply calls all injected ItemWriters for every item, you'll need to implement the dispatching logic yourself in your own composite item writer.

[Jainendra]

Thanks for your reply. But I have a issue with this

code in CompositeItemWriter is
public void write(List<? extends T> item) throws Exception {
for (ItemWriter<? super T> writer : delegates) {
writer.write(item);
}
}

I can understand what you are trying to say is, here I can use logic to dispatch to different writer.

Problem 1: How I can identify the ItemWriter. Say I have three writer ItemWriter1, ItemWriter2, ItemWriter3 so I will idendify the writer class.

Problem2: I did some ground work . I have written a configuration for composite writer in job.xml file

<bean id="compositeWriter"
class="org.springframework.batch.sample.domain.ord er.internal.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="fileItemWriter" />
<ref bean="fileItemWriter1" />
</list>
</property>
</bean>

Now I have defined two differend ItemWriter
<bean class="FlatFileItemWriter" id="fileItemWriter">
<bean class="FlatFileItemWriter" id="fileItemWriter1">

And I have specified different file name to it.

So if we see the compositeWriter.write(). It should write same data to both the file.
But what I found was, It was writing twice to one file and is not writing anything on the second file.

I ran the debug and found it is trying to invoke different FileItemWriter, but it write twice to a single file, instead of writing line to both the file.

Will appreciate if you can explain what is happening. I tried a lot to find the reason and write to both the files.

[Jainendra]

I tried to use composit writer with old version and it works fine. But when I tried using org.springframework.batch.dist-2.0.0.M3-with-dependencies.zip there is a problem with compositeItemWriter. There is some bug I guess.

Regards

[robert.kasanicky]

I don't see how the composite writer can cause such trouble given how simple it is. We still have the compositeItemWriterSampleJob that works fine. If you were able to recreate the problem there that would be very helpful.

[fall2spring]

I guess, you need to provide the definition of the following
beans as well. Are you sure that by any chance, you are not
mentioning same file name as the property to both the writers ?

<bean class="FlatFileItemWriter" id="fileItemWriter">
<bean class="FlatFileItemWriter" id="fileItemWriter1">

I've used spring batch M3 in similar capacity and don't see any problem.

Thanks.

[Jainendra]

compositeItemWriterSampleJob works because it
1. writes to a file
2. insert data to database.

But the problem is with writing to two file.

I am giving different file names so there is no confusion with the file name. As I said , I tried with the spring-batch-dist-2.0.0.M1 and earlier version and it is working fine. But with spring-batch-dist-2.0.0.M2 and spring-batch-dist-2.0.0.M3 there is a problem.
The configuration is below
Regestring the stream

	<bean id="compositeItemWriterJob" parent="simpleJob">
		<property name="steps">
			<bean id="step1" parent="simpleStep">
				<property name="streams">
					<list>
						<ref bean="fileItemReader" />
						<ref bean="fileItemWriter" />
						<ref bean="fileItemWriter1" />
					</list>
				</property>
				<property name="itemReader" ref="fileItemReader" />
				<property name="itemProcessor">
					<bean class="org.springframework.batch.item.validator.ValidatingItemProcessor">
						<constructor-arg ref="fixedValidator" />
					</bean>
				</property>	
				<property name="itemWriter" ref="compositeWriter" />
			</bean>
		</property>
	</bean>

defining composite writer

	<bean id="compositeWriter"
		class="org.springframework.batch.item.support.CompositeItemWriter">
		<property name="delegates">
			<list>
				<bean
					class="org.springframework.batch.sample.domain.trade.internal.TradeWriter">
					<property name="dao" ref="tradeDao" />
				</bean>
				<ref bean="fileItemWriter" />
				<ref bean="fileItemWriter1" />
			</list>
		</property>
	</bean>

defining the beans

	<bean class="org.springframework.batch.item.file.FlatFileItemWriter"
		id="fileItemWriter">
		<property name="resource"
			value="file:target/test.TEMP.txt" />
		<property name="lineAggregator">
			<bean
				class="org.springframework.batch.item.file.transform.PassThroughLineAggregator" />
		</property>
	</bean>

	<bean class="org.springframework.batch.item.file.FlatFileItemWriter"
		id="fileItemWriter1">
		<property name="resource"
			value="file:target/test-outputs/test.TEMP1.txt" />
		<property name="lineAggregator">
			<bean
				class="org.springframework.batch.item.file.transform.PassThroughLineAggregator" />
		</property>
	</bean>

[robert.kasanicky]

I've tried adding another FFIW into the composite sample and ran into the same problem. I'm not sure where things go wrong yet, but you can track the issue here http://jira.springframework.org/browse/BATCH-969

[Jainendra]

I have found the error for ComositeWriter.
If I set the TransactionSynchronizationManager.isActualTransact ionActive() as false then it works.

It means there is a issue with the use of transaction.

In class OutputState we create new instance of FileOutputStream and encapsulate with
TransactionAwareBufferedWriter to get outputBufferedWriter

Code: TransactionAwareBufferedWriter outputBufferedWriter = new TransactionAwareBufferedWriter(Channels.newWriter( fileChannel, encoding));

So what we do is we keep on saving the lines read in a transaction in buffer with key
BUFFER_KEY.

And at the end we call commit using code
transactionManager.commit(transaction); in code TaskletStep.java .
So here there is some issue with binding both the Writers and the transaction

Thanks for jira

[robert.kasanicky]

Yes, it's definitely the TransactionAwareBufferedWriter causing the issue.