Does Spring Batch Support Multiple I/O files

[aslam.shahulhameed]

Hi Dave,

Does spring Batch support multiple I/O files in a single jobRun for a particular scheduleDate.

Scenario:
Read 100 rows from the database and write to file, Output file can have atmost 50 lines(rows). So does spring batch support creating 2 files each containing 50 lines (filename.txt.001 & filename.txt.002).

[lucasward]

This is supported, however, there isn't any kind of 'max record count' that would signal to the FileOutputSource that a new file should be created. The outputsource would simply need to be closed and reopened. However, a wrapper could easily be written that would do this for you.

[Dave Syer]

http://opensource.atlassian.com/proj...rowse/BATCH-34

[Jainendra]

Is it possible to write to multiple files in one go? Say by putting to seperate writers?
My case I have one input file and 2 output files with some different processing in between.

Thanks
Jai

[Dave Syer]

People do that all the time by writing a composite ItemWriter. Tjere is almost always business / application logic to decide which writer to send data to, so we don't do this in the framework. You could also consider using Spring Integrationt to route the data to the correct writer.

[Jainendra]

What if we HAVE to write data in multiple files simultaneously

As below I have added two writer
<bean id="compositeWriter"
class="org.springframework.batch.sample.domain.ord er.internal.CompositeItemWriterT">
<property name="delegates">
<list>
<ref bean="fileItemWriter" />
<ref bean="fileItemWriter1" />
</list>
</property>
</bean>

fileItemWriter - writes to File1
fileItemWriter1 - writes to File2

The logic in composite writer is

public void write(List<? extends T> item) throws Exception {

for (ItemWriter<? super T> writer : delegates) {
writer.write(item);
}
}


In this case the writer is writing twice in the same file File1. And File2 remains blank

Thanks

[Dave Syer]

Did you register both delegates as ItemStreams (see property streams in *StepFactoryBean)?

(Please use [code][/code] tags to post code and stack traces.)

[Jainendra]

Thanks I will take care of using the code syntax

Were will I register the delegates as InutStreams. Because my code is

<property name="steps">
   <bean id="step1" parent="simpleStep">
   <property name="streams">
   <list>
      <ref bean="fileItemReader" />
      <ref bean="fileItemWriter" />
      <ref bean="fileItemWriter1" />
   </list>
   </property>
   <property name="itemReader" ref="fileItemReader" />
   <property name="itemProcessor">
      <bean class="org.springframework.batch.item.validator.ValidatingItemProcessor">
          <constructor-arg ref="fixedValidator" />
      </bean>
    </property>	
    <property name="itemWriter" ref="compositeWriter" />		
   </bean>
</property>

The other code is the one I posted in my last post configuring compositeWriter and configuring the flatFileItemWriter.

What I found is. When i ran the same test with 1.0.0 release it is working properly writing the output to both the file. I can understand there is some configuration that I will have to do with InputStream, but unable to figure this out, as I even tried to check the API and the documentation

Regards
Jainendra

[Jainendra]

Hi,
I found out something on this. I tried to run the same code with spring-batch-dist-2.0.0.M1 and it works fine.
But with spring-batch-dist-2.0.0.M3 it is saving data twice in one file instead of writing to both the files.
Is there some issue with spring-batch-dist-2.0.0.M3? or do I need to make some configuration changes

Thanks

[Dave Syer]

Your config looks fine. Must be a bug. Do you want to open a JIRA to track it?