Setup question

[Reijnemans]

Hi All,

Could you give me some advised how to setup the following case with spring batch:

I've have to read several types of objects from the database and I have to write those object to a single xml file.

DB                                                      XML
 |            Locations                                  |
 |----------------------------->|                        |
 |            Vehicles          |                        |
 |----------------------------->|----------------------->|
 |            Movements         |                        |
 |----------------------------->|                        |

so you could say I've multiple readers where each reader gives a different type of object and one writer

Now I wondering what the best thing is todo:

* Should I use multiple steps (reader/writer) to read all data from the database put it somewhere in the cache. And let the last reader/writer, read the cache and write to the file. In this case where should I store the cache? I don't want to those objects in the database? but other job(data/params) have to be stored in the database?

* Write a composite reader (but I'm not sure if this is possible for my case)?

* something else?

Regards,

D.Reijntjens

[tiger.spring]

Looks like you want to read from many tables and write to one file.

Isn't it possible to write SQL joins to fetch data from these table and then write to xml. If they are being written in one xml file, I assume it is relational data.

else you may also write a stored proc and use StoredProcedureItemReader.

[Reijnemans]

Thanks for your response tiger.spring.

Unfortunally it is not possible to write it in one query. I already looked at it. And a storedProcedure is not allowed.

Think the best thing is to have a open stream for multiple writer, Or a second jobcontext one in memroy and one database. however this is not possible.

Is it possible to keep something in memory during the job besides the (job/step)execution-context.

Regards,

D.Reijntjens

[tiger.spring]

I don't think you should use multiple writer because you want to create one xml file which could be something like:

<Customer>
    <name>
    <Address>
        <street>
        <state>
     </Address>
    <Order>
          <id>
          <products>
                   <id>
                   <qty>
           </products>
     </order>
</customer>

You should still be able to write one customItemReader which utilizes a composite service and returns each customer object to the item readers read() method and then write it using xml writer.

[Reijnemans]

I don't think you should use multiple writer because you want to create one xml file which could be something like:

Correct I prefer not to.

You should still be able to write one customItemReader which utilizes a composite service and returns each customer object to the item readers read() method and then write it using xml writer.

This is maybe possible, but I'm not totally sure how a composite service is working. As I read a ItemReader is calling another ItemReader.

I've have to read one "schedule" (root element) in the schedule is a list of locations, vehicles. movements, ....

Some of the data (Movements for example) we read about 10.000 records from the database, perform some business logic, and take a small peace of the movement we have to put in the XML. At first I wanted todo this with a HibernatePagingItemReader or HibernateCursorItemReader, to make sure we don't have all Movements in memory, and the transactions doesn't take to long. Do I still have this functionality as I use a composite reader? Or does a composite reader (In my case) read all data for the root element at once in one transaction?

[Reijnemans]

Maybe I have to write all data to ThreadLocal, and when everything is read, create a reader on my ThreadLocal and write it to an xml file.

[tiger.spring]

Correction: I was referring to composite service and not composite reader.

Your service will be framework independent and will return list of customer objects to the reader.

Correct I prefer not to.



This is maybe possible, but I'm not totally sure how a composite service is working. As I read a ItemReader is calling another ItemReader.

I've have to read one "schedule" (root element) in the schedule is a list of locations, vehicles. movements, ....

Some of the data (Movements for example) we read about 10.000 records from the database, perform some business logic, and take a small peace of the movement we have to put in the XML. At first I wanted todo this with a HibernatePagingItemReader or HibernateCursorItemReader, to make sure we don't have all Movements in memory, and the transactions doesn't take to long. Do I still have this functionality as I use a composite reader? Or does a composite reader (In my case) read all data for the root element at once in one transaction?