We have few batch processes that require us to write to flat files more than once, and when I looked up appending to text files in the forum I found an old post
Where Dave stated that
I was wondering if in the latest version of Spring batch that's still the case.
file item writers intentionally only append on restart
If so, then I think there're cases where we need and in our case we have to append to an existing flat file.
To give more details here's what we are trying to do:
- read a route data
- Process the route and its collocated routes (children)
- write each route (the parent and children routes) to their designated files.
The need to write to the same route file is because lot of routes are partially collocated, which means some of the route records will be processed by themselves and some will be processed with the primary route( the parent) they are collocated with.
Also the choice to write to flat files is done to avoid the overhead of DB updates. We are handling over 10 million records. The flat files will be loaded later into the DB using sqlLoader.
So Do we need to extend the itemWriter to accommodate for append (like the person in the above post was testing) or is there a work around that's there that we are not aware of?
Thanks in advance for your help