May 25th, 2011, 01:52 AM
MapReduce like operations in SpringBatch step
Is it possible to do MapReduce style operations in SpringBatch?
I have two steps in my batch job. The first step calculates average. The second step compares each value with average to determine another value.
For example, Lets say i have a huge database of Student scores. The first step calculates average score in each course/exam. The second step compares individual scores with average to determine grade based on some simple rule:
1. A if a student scores above average
2. B if a student score is Average
3. C if the student scores below average
Currently my first step is a Sql which selects average and writes it to a table. Second step is a Sql which joins average scores with individual scores and uses a Processor to implement the rule.
There are similar aggregation functions like avg, min used a lot in Steps and I'd really prefer if this can be done in Processors keeping the Sqls as simple as possible. Is there any way to write a Processor which aggregates results across multiple rows based on a grouping criteria and then Write Average/Min to the Output table once?
Tags for this Thread