Returning statistics to the calling application

[alpheratz]

My batch has a job with multiple steps...the 'important' step in the middle (the one that does the 'real' work, so to speak) reads a record, and writes it to either a 'good' XML file or a 'bad' csv file depending on whether it passes validation.

The calling application needs to know how many were read, how many were written to the good file and how many were written to the bad file.

Each of the output files has a writer (of course), so can I get hold of their statistics somehow?

If not, what is the recommended way of dealing with this?

Cheers,

Alph

[lucasward]

It depends upon how you've implemented validation failure. If you throw an exception and write out to the csv in a skip listener, then you can use writeCount and readSkipCount (or processSkipCount, depending) You could also not through an exception, write out to the csv in the ItemProcessor, and then return null, to indicate a 'filtered' item. You can then use writeCount and filterCount from StepExecution. The later would be the way I would do it.

[alpheratz]

Thanks for the prompt reply, Lucas.

I saw writeCount, etc. in the doco. and I have just put 2 + 2 together.

I can do:

jobExcution.getStepExecutions() and look at the individual statistics:

StepExecution: id=0, name=startupStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
StepExecution: id=1, name=processStep, status=COMPLETED, exitStatus=COMPLETED, readCount=1000, filterCount=0, writeCount=656 readSkipCount=0, writeSkipCount=0, commitCount=1001, rollbackCount=344, exitDescription=
StepExecution: id=2, name=endingStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=

Good enough for my immediate needs.

BUT: what if I wanted to hold a count of (say) all records over n bytes, or all with the word 'Alpheratz'...this is the general case...how would I maintain and return this sort of app-specific info?

Would I set up a filter for each situation, chain them together and use filtercount? A bit 'indirect', although workable. Isn't there a sort of 'global' environment available?

Apologies if this is a naieve question...

Cheers,

Alph

[lucasward]

For a specific count, you need to store the value within the ExecutionContext, which Spring Batch persists at commit points. Make sure you only update it in the update method of ItemStream, in order to avoid potential issues with rollback.