Processing apx. 500000 db inserts in a batch job

[geira]

We have a task where we need to iterate over a resultset on apx. 250000 and perform apx 2 db inserts for every record read.

Which approach should we take. We have tried using a RestartableItemProviderTasklet with an inputsource providing a record on each iteration, calling a method performing 2 db inserts.

Are there any more effective way to do this. How could we make use of Spring jdbc and batchUpdate?

Should we use another batch implementations which would be more effective?

Rgds.

Geir A.

[Dave Syer]

There is a BatchSqlUpdateItemWriter that you might find useful as a starting point. You might even just be able to use two of them, one for each insert. I can see from the old terminology you are not using an up to date release, so the first thing you might want to do is upgrade. I'm not sure off the top of my head if the SQL update writer is in m5 or only in the more recent snapshots, but it should be easy to follow. The key thing, if you are implementing your own writer, is to use the flush callback to execute the JDBC batch.