Spring batch header and footer to flat file writer

Hi all,

I am a Spring Batch newbie. There are two parts to my question. I feel a bit silly with the first one, because it feels like there is probably an easy answer to this....I am just missing somethig and this is probably related to my lack of in-depth knowledge of Spring Batch.

I have records in a database and need to write them to a file. No problems here. I need a header and footer written to this file as well. It needs to be summary header and footer. Both the header and footer needs to have the total number of recods in. The footer also needs the total amount (each record has transaction amount).

So my questions:
1. I found an example of the summary footer. I tried to implement it. The footer gets written, but the amount wasn't updated. It looks like he "write" method is never reached. I have never written a custom reader and writer. I thought that maybe the class="" part should change to point to my new writer class, but then the property fields don't work anymore?

2. I have no idea how to do a summary header.

Code for part one of my question:

Code:

---

<beans:bean id="ItemReader.ExportDataJob" class="org.springframework.batch.item.database.JdbcCursorItemReader">
        <beans:property name="dataSource" ref="DataSourceBean.DEV.mhdev1"/>
        <beans:property name="sql" value="SELECT * FROM TEST WHERE STATUS=2"/>
        <beans:property name="rowMapper">
            <beans:bean class="za.co.discovery.mapper.ValidatorFieldSetMapper"/>
        </beans:property>
    </beans:bean>

    <beans:bean id="FooterWriter.ExportDataJob" class="com.TrailWriter">
        <beans:property name="delegate" ref="ItemWriter.ExportDataJob"/>
    </beans:bean>

    <beans:bean id="ItemWriter.ExportDataJob" class="org.springframework.batch.item.file.FlatFileItemWriter">
        <beans:property name="resource" value="file:${data.extract.file.path}/${data.extract.file}_Export.txt"/>
        <beans:property name="lineAggregator">
            <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                <beans:property name="delimiter" value="|"/>
                <beans:property name="fieldExtractor">
                      <beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                          <beans:property name="names" value="aval,bval,cval,amount"/>
                      </beans:bean>
                </beans:property>
            </beans:bean>
        </beans:property>
        <beans:property name="footerCallback" ref="FooterWriter.ExportDataJob"/>
    </beans:bean>

---

The custom writer (I only implemented the total for the amount so far. I will just add a counter for the other part needed for the footer).

Code:

---

public class TrailWriter implements ItemWriter<ValidatorDto>, FlatFileFooterCallback{

    private ItemWriter<ValidatorDto> delegate;
    private double totalAmount = 0.0;

    public void write(List<? extends ValidatorDto> items) throws Exception{
        double chunkTotal = 0.0;
        System.out.println("****THIS CODE WAS ENTERED*********");

        for (ValidatorDto dto: items){
            chunkTotal += Double.parseDouble(dto.getTransactionAmount());
        }

        delegate.write(items);

        totalAmount = totalAmount + chunkTotal;

    }

    public void writeFooter(Writer writer) throws IOException{
        writer.write("CheckSum: " + totalAmount);
    }

    public void setDelegate(ItemWriter delegate){
        this.delegate = delegate;
    }

}

---

I don't see anything obviously wrong with what you have here to generate a summary footer. That being said, you didn't include the configuration of your job itself so I don't know if you've configured the use of these readers correctly.

With regards to writing a "summary header", Spring Batch doesn't offer anything within the framework to handle something like that. The best you could do is to store the information for the header in the ExecutionContext and in another step, copy the file generated adding the header.

An alternative to copying the file would be to use a FileChannel to insert the header at the top of the file in that second step. I haven't tried this but I don't see why it wouldn't work. The only risk here is that if this second step fails, you end up with a corrupted file instead of one good one and one bad one.

Thanks.

Footer: I looked at the job configuration. I pointed to the ItemWriter and not the FooterWriter. Changed this. At least I can see it is entering my code now, but now I get the following error. Is it still a configuration problem?

Code:

---

org.springframework.batch.item.WriterNotOpenException: Writer must be open before it can be written to

---

Still looking into the suggested solutions for the Header

FlatFileItemWriter (and many other ItemWriters) implement ItemStream and need to be opened prior to being used. I would suggest your ItemWriter implement ItemStreamWriter and in the open/update/close methods, see if the delegate is an ItemStream. If it is, delegate the call to the appropriate method.