Item writer with per item and per commit writes?

[gph]

Hi,

This is my first post so apologies if I accidentally break any rules.

I'm wanting to implement a Batch job using a batch chunk consisting of a reader, processor and writer as follows:

Read from DB -> process -> send email and write to DB.

My understanding is that each of these are on a per item basis. So in the above scenario a single DB read would be made, but the reader presents the data from each result row in turn to the processor for processing each item, then on to the writer when the commit limit is reached, which is fine.

My issue is with the writer in that I would like to send an email for each item, but would prefer to do a single DB update for all items in one go rather than for each item.

i.e. a single "UPDATE ... WHERE id IN (1,2,3)" instead of:
"UPDATE ... WHERE id = 1;
UPDATE ... WHERE id = 2; etc."

For this I'm planning to do implement a CompositeItemWriter with something similar to the following:

public void write(java.util.List<SomeItem> item) {
    int[] ids = int[item.size()];
    int i = 0;
    for (SomeItem o : item) {
        // do single item processing i.e. send email
        int[i] = item.getId();
        i++;
    }
    // do single DB update using ID array/list
}

Just wondering if I'm on the right track, or whether there's a better recipe for this? (Or whether I'm missing something entirely).

Thanks in advance,
G.

[gph]

For this I'm planning to do implement a CompositeItemWriter...

Actually, a standard ItemWriter implementation would suffice but the question regarding the general approach still stands.

[Marten Deinum]

That is not what the CompositeItemWriter is intended for. The CompositeItemWriter is to use multiple ItemWriters to handle the item, not to be overriden by yourself.

Next to that are you sure you want to do it in the way you are currently thinking. Now if you have send 25 emails and your update fails, then what? No way to rollback your emails. I would do it the other way around, first do the update and in a next step send the emails.

[gph]

Thanks for your reply.

That is not what the CompositeItemWriter is intended for. The CompositeItemWriter is to use multiple ItemWriters to handle the item, not to be overriden by yourself.

See secondary post above regarding not using the composite writer.

Next to that are you sure you want to do it in the way you are currently thinking. Now if you have send 25 emails and your update fails, then what? No way to rollback your emails. I would do it the other way around, first do the update and in a next step send the emails.

A moot point for sure, although what if I do the update and the emails fail given the update really just says that emails were sent (in this case)? Anyway, it's relatively by-the-by as regardless of what's actually been done, I'm more curious as to whether this is the right strategy from a design perspective when wishing to do a write on a per item basis as well as a write on completion of a bunch of items being processed.

Cheers.

[Marten Deinum]

Resending the emails is easier done than processing the updates again imho... Hence the suggestion to make it a seperate step, that way you could easily retry it.

[gph]

Point taken. Thanks.

Is this the general approach/place one would put per item and per commit processing though, or is there a better way?