Manual chunk commit

[Dave Syer]

Could you peek ahead and try to detect the invalid group (as per the original post in this thread), then force a commit using a completion policy and deal with marking incomplete groups in a listener, or possible the writer. Then the next chunk will start with the peeked item (which you know is part of the next group)?

Or you could do the group validation as a completely separate step?

[LukeL99]

Could you peek ahead and try to detect the invalid group (as per the original post in this thread), then force a commit using a completion policy and deal with marking incomplete groups in a listener, or possible the writer. Then the next chunk will start with the peeked item (which you know is part of the next group)?

Or you could do the group validation as a completely separate step?

The group validation as a separate step is an interesting idea that we hadn't thought of. That might work, although right now we're working on a hack to get it working before we go through and completely rewrite the batch job for a new log format.

Thanks for your help.