Summarizations of ItemWriter outputs in parallel job processing

[zytan]

Here's my scenario:

Under Competing Consumers pattern (http://www.enterpriseintegrationpatt...Consumers.html)
The Sender sends request message chunks to consumers, each consumer is implemented as Spring Batch Parallel Job, and after processing the request chunk in multiple threads, Consumer needs to reply the chunk process result to Sender.

So here comes my question: In a Spring Batch Parallel Job, how can we summarize process results from multiple concurrent ItemWriters?

What I can think out is this way:
Each item writer thread writes its process result to database table, and then create another step for this job to collect the information from db and then reply it to the sender. But creating a table for it a quite a overhead in my case.....

Does anyone have any good solution?

btw: I am using S.B 1.0.0.FINAL

[adrianshum]

I think the missing link here is Spring batch does not yet provide any way to get the 'result' of the job, probably because in most batch processing, there are no meaningful 'processing result' apart from the exit code.

You may consider having an object holder (a map, or a list) in your job's app context, and your tasklet/writers put the 'result' of your processing in it and, upon completion of job, get the result object from app context. (Is it a valid way?)

In fact I am going to do something similar to yours (making use of JMS and distribute parallel sub-jobs to multiple machine). Luckily, I don't need to consolidate results from my parallel sub-job (I am only interested on whether the job completed successfully or not)

[zytan]

Hi Adrian, by 'app context' do you mean Execution Context of Spring batch?

You may consider having an object holder (a map, or a list) in your job's app context, and your tasklet/writers put the 'result' of your processing in it and, upon completion of job, get the result object from app context. (Is it a valid way?)

[adrianshum]

Hi Adrian, by 'app context' do you mean Execution Context of Spring batch?

no no... I mean the Spring Application Context which contains Spring Batch related beans.