Jan 14th, 2011, 09:40 AM
Could you peek ahead and try to detect the invalid group (as per the original post in this thread), then force a commit using a completion policy and deal with marking incomplete groups in a listener, or possible the writer. Then the next chunk will start with the peeked item (which you know is part of the next group)?
Or you could do the group validation as a completely separate step?
Jan 14th, 2011, 09:58 AM
The group validation as a separate step is an interesting idea that we hadn't thought of. That might work, although right now we're working on a hack to get it working before we go through and completely rewrite the batch job for a new log format.
Originally Posted by Dave Syer
Thanks for your help.