Job Statistics

[johny_z]

What are the various job statistics details provided by spring batch?
Can i add additional statistical data apart from those spring batch provides?
For eg: if i have a job has to reads 100 records from a table and it fails at 50th record. So when i run the job again, will it start from where it failed?
Should this be handled programmatically or can spring batch handle this?

[lucasward]

Spring batch keeps basic statistics about the run. For example, commitCount, rollbackCount, readSkipCount, itemCount, etc.

The second part of your question is actually different from statistics. Storing state information for the purposes of starting where you left off again in the case of failure is what we call restart information. All state is stored in an ExecutionContext. When an ItemStream is opened, it is given the ExecutionContext for the step. If it's just starting, the context will be empty, if it's a restart, there will likely be state information in it from the previous run. (assuming something was put in it)

The following section in the reference guide should help explain the concepts to you:

http://static.springframework.org/sp...e.html#d0e2959

[johny_z]

Thank you for your reply. I have a few more doubts.
1. Does spring batch provides any reporting feature? Suppose if i executed a job that has many steps and i want to see the job statistics report of the job in detail for each steps in the job?
Or should i get the details from meta data schema tables programmatically?

2. I could found out the commitCount and itemCount in metadata schema tables, but where other details like rollbackCount, readSkipCount etc will be stored?

3. What are the different ways to stop a job? For eg: if i have a job with three steps. I invoked this job by a scheduler and i want to stop the job in the middle of execution of second step . Will the spring batch take care of finishing the second step execution properly? What will happen if i re-run the job? Will it start from first step again or from the third step?

[Dave Syer]

1. Does spring batch provides any reporting feature?

Not in 1.x, but as you said, you can always point any tool that understands database tables at the Spring Batch metadata. 2.0 has a JobExplorer interface (current name), which isn't really "reporting" (fancy formatting, aggregation, PDF etc.), but does gove you read-only access to the metadata.

2. I could found out the commitCount and itemCount in metadata schema tables, but where other details like rollbackCount, readSkipCount etc will be stored?

What version are you looking at? All those fields are stored in the step execution table in 1.1 and above.

3. What are the different ways to stop a job?

See the user guide section on launching a job (http://static.springframework.org/sp...n.html#d0e3476).

For eg: if i have a job with three steps. I invoked this job by a scheduler and i want to stop the job in the middle of execution of second step . Will the spring batch take care of finishing the second step execution properly? What will happen if i re-run the job? Will it start from first step again or from the third step?

It will stop where you ask it to (i.e. in the second step in your example). As long as your stateful components implement ItemStream (and are registered with the step) the second step will be completed on restart. See the user guide (same chapter, for example http://static.springframework.org/sp...n.html#d0e3476).

[johny_z]

Thanks a lot.
Can a spring batch job be scheduled using Tivoli Worflow Scheduler?
Is there any examples available for this?

[Dave Syer]

No reason why not, but you will have to do some work to get your stop and monitoring use cases working in 1.x. Most people using non-Java schedulers like Tivoli just use the CommandLineJobLauncher, which doesn't give you any access to the running JobExecution. 2.0 will make it easier (e.g. see JobOperator). In the meantime you would have to expose a remote endpoint (JMX, web service etc.) that could be contacted by Tivoli and keep n the a reference to the JobExecution internally in the server. The JMX sample does this.

There are no samples using Tivoli per se (or any other proprietary tool), but the CommandLineJobLauncher is used extensively and has plenty of documentation. There is also a Websphere Performance Pack in the pipeline from SpringSource which probably will include Tivoli features. It's not available yet, and the feature set is not finalised, but contact sales@springsource.com if you want to stay in the loop.

[johny_z]

Thank you.
When i ran the QuartzBatchLauncher.java in the samples, i got the following error. But when executed adhocLoopJob.xml and footballJob.xml individually, both are executed properly.(I am using orcle db for batch meta-data tables instead of in-memory db; property files are there in the classpath also). Would you please help me out...

Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

No reason why not, but you will have to do some work to get your stop and monitoring use cases working in 1.x. Most people using non-Java schedulers like Tivoli just use the CommandLineJobLauncher, which doesn't give you any access to the running JobExecution. 2.0 will make it easier (e.g. see JobOperator). In the meantime you would have to expose a remote endpoint (JMX, web service etc.) that could be contacted by Tivoli and keep n the a reference to the JobExecution internally in the server. The JMX sample does this.

There are no samples using Tivoli per se (or any other proprietary tool), but the CommandLineJobLauncher is used extensively and has plenty of documentation. There is also a Websphere Performance Pack in the pipeline from SpringSource which probably will include Tivoli features. It's not available yet, and the feature set is not finalised, but contact sales@springsource.com if you want to stay in the loop.

[johny_z]

Thank you.
When i ran the QuartzBatchLauncher.java in the samples, i got the following error. But when executed adhocLoopJob.xml and footballJob.xml individually, both are executed properly.(I am using orcle db for batch meta-data tables instead of in-memory db; property files are there in the classpath also). Would you please help me out...

Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

[johny_z]

I got the following error when i ran QuartzBatchLauncher.java in the samples:
Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

But when I ran "jobs/adhocLoopJob.xml" and "jobs/footballJob.xml" individually, both are executed successfully. I used oracle for meta-data tables instead of in-memory db. Property files are there in the classpath.

Would you please help me out...?

[Dave Syer]

From batch-oracle.properties in the samples project:

batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler

Did you include that file in your context (the easiest way is by setting a System property environment=oracle)?

Actually the lob handler is only there to support very old versions of Oracle, and we'll be removing it from the samples at some point anyway. If you have Oracle 10g or above you don't need it so you can root out the bean definition that is causing problems and delete it.