How to process logically related rows after ItemReader in Spring Batch?

I have already posted this question on StackOverflow. But I want to be part of this forum as well see If I get more answers overall. So far, I got no answer yet on SO.

Scenario

To make it simple, let's suppose I have an ItemReader that returns me 25 rows.

The first 10 rows belong to student A

The next 5 belong to student B

and the 10 remaining belong to student C

I want to aggregate them together logically say by studentId and flatten them to end up with one row per student.

Problem

If I understand correctly, setting the commit interval to 5 will do the following:

Send 5 rows to the Processor (which will aggregate them or do any business logic I tell it to).
After Processed will write 5 rows.
Then it will do it again for the next 5 rows and so on.
If that is true, then for the next five I will have to check the already written ones, get them out aggregate them to the ones that I am currently processing and write them again.

I personally do no like that.

What is the best practice to handle a situation like this in Spring Batch?

Alternative

Sometimes I feel that it is much easier to write a regular Spring JDBC main program and then I have full control of what I want to do. However, I wanted to take advantage of of the job repository state monitoring of the job, ability to restart, skip, job and step listeners....

If I understand what you are trying to do, what you really want is an ItemReader that aggregates the data on input. This can be done a number of ways depending on the type of input you are using (File vs DB vs etc). By doing your input this way, you'll set your commit count to a number and that number will be the number of student ids to process/write.

Hi Michael,

I though of putting the logic to flatten the rows and produce 1 per student in the reader, but I am not sure if that is the best practice. I thought that business logic should go into the processor. If I put the logic at the reader, then I would not need a processor at all, I would just pass it to a writer correct?

When considering how to structure your ItemReader/ItemWriters, you need to determine what is considered an item. An item, from the point of Spring Batch is a single element to be processed. In your case, an item sounds like it is the flattened object. Because of this, it is perfectly ok to structure your ItemReader to build this object as you need.

With regards to whether or not you need an ItemProcessor if you build your ItemReader that way depends on what you need to do with the item once it's built. Let me know if that makes sense.

Thanks a lot that makes complete sense.