Writing data to Multiple file

[harit.dhiman]

I have one scenario-
1) Reader read
a) Company data from database.
b) Add this data to CompanyDTO.
c) Add this DTO to ListItemReader.

This CompanyDTO is then sent to writer one by one to the writer.
At the writer side corresponding to each CompanyDTO a separate Flat file has to be created, example companyData-01.txt, companyData-02.txt etc... \the number of txt file will depend on the number of CompanyDTO. i.e. corresponding to each DTO a separate file needs to be created.

I tried using Composite Item Writer in order to print this data to a flat file using FlatFileItemWriter but each and every time one file is getting generated and all DTO data is coming in one file instead of separate file.

[DHGarrette]

Are you using the built-in CompositeItemWriter class? That one is intended to write the same thing to every file. You need to create a custom composite ItemWriter that delegates the right information to the right file.

[harit.dhiman]

Are you using the built-in CompositeItemWriter class? That one is intended to write the same thing to every file. You need to create a custom composite ItemWriter that delegates the right information to the right file.

The output file needs to be created dynamically. Its not that i am having 2 FlatFileItemWriter and data should be printed to these file. File should be created dynamically. I am using my own composite itemwriter and using delegator to print data to file but all data is getting printed to one file. New file is not getting created.

At the writer side corresponding to each CompanyDTO a separate Flat file has to be created, example companyData-01.txt, companyData-02.txt etc... the number of txt file will depend on the number of CompanyDTO. i.e. corresponding to each DTO a separate file needs to be created.

[DHGarrette]

Oh, I see now. Unfortunately it's a bit hard to see what's wrong without the code. Can you post your composite ItemWriter?

[harit.dhiman]

Oh, I see now. Unfortunately it's a bit hard to see what's wrong without the code. Can you post your composite ItemWriter?

I dont have code right now. I can try to explain what we are doing -
1) DTO has List<attribsA>, attribB. Now in order to print List<AttribsA> we have a FlatFileItemWriter. And in order to print attribB object we have another FlatFileItemWriter. Both of these writer are pointing to same output file.
Now in write method if i try to open the related writers, write the data and try to close the writers, i am getting error something like 'writer already closed'.
My main query is:
How can i create multiple output file with different names such as companyData-01.txt, companyData-02.txt etc....

[DHGarrette]

Both of these writer are pointing to same output file.

This won't work. Each file should have exactly one associated ItemWriter. However, there should be no problem sending both objects to the same ItemWriter.

Now in write method if i try to open the related writers, write the data and try to close the writers, i am getting error something like 'writer already closed'.

This seems fine in principle, so it must be an issue with your code. The write() method will be like this:

int fileNumber = 0;

write(List<? extends MyDTO> items) {
    for(MyDTO item : items) {
        FlatFileItemWriter<Object> w = new FlatFileItemWriter<Object>();
        w.setResource(baseFilename + (++fileNumber));
        w..... //set other properties
        w.open(new ExecutionContext());
        for(AttribsA a : item.getAttribsA()) {
            w.write(a);
        }
        w.write(item.getAttribB());
        w.close();
    }
}

[harit.dhiman]

Thanks...I will check this out and update you respectively.

[harit.dhiman]

This won't work. Each file should have exactly one associated ItemWriter. However, there should be no problem sending both objects to the same ItemWriter.


This seems fine in principle, so it must be an issue with your code. The write() method will be like this:

int fileNumber = 0;

write(List<? extends MyDTO> items) {
    for(MyDTO item : items) {
        FlatFileItemWriter<Object> w = new FlatFileItemWriter<Object>();
        w.setResource(baseFilename + (++fileNumber));
        w..... //set other properties
        w.open(new ExecutionContext());
        for(AttribsA a : item.getAttribsA()) {
            w.write(a);
        }
        w.write(item.getAttribB());
        w.close();
    }
}

I tried above changes but still i am getting the error message. Please find attached
1) error.txt for error message.
2) writer.txt - Containing writer snippet.
3) xml snippet.txt - Step configured in batch specific xml file. recordAAttribsWriter and recordBAttribsWriter are two flat file item writer. In the example we are reading fileName from property file and in writer i am appending it with date (Its just for example, to have distinct file name), but still i am getting the attached error message.

[DHGarrette]

recordAAttribsWriter.setResource(resource);
recordBAttribsWriter.setResource(resource);

As I mentioned before, you cannot have more than one writer for a single file. You either need to have a different file, OR use the same writer for both As and Bs.

[harit.dhiman]

As I mentioned before, you cannot have more than one writer for a single file. You either need to have a different file, OR use the same writer for both As and Bs.

I modified the code. Used one writer...but still i am getting same error...Stream already closed. Is it a bug in Spring framework that we cannot have multiple out put files....Please suggest?