Forcing the step to finish when JobOperator.stop is called on the job

[gerson721]

Hi,

I'm wondering if there's a way to force the step execution to finish when I call the JobOperator's stop method. From the tests that I ran, the StepExecution only finds out about the JobInterruptedException after the step has finished executing.

Another issue is the current step that is being executed when the stop method is called, gets run again when the job is restarted even if it has actually finished running as a consequence of the first issue I raised above. Is there a way of telling spring batch not to rerun the last step that has finished running?

Thanks.

[gerson721]

Hi again,

Nobody has responded to my post so I thought of giving more info on what I was trying to achieve...

I've got a job that is to transfer a number of files from a remote source to a remote destination. Let's say, the job is to transfer 3 big files and each job step will transfer one file at a time. Assume that each file is 50GB in size and we've already finished transferring the first file. The second file starts transferring when the job receives a stop signal from the JobOperator.

Ok, here's the thing, spring batch still lets the second step finish processing before it actually throws a JobInterruptedException. What I've noticed as well is that from within the step, there's no way for me to know that either the job or step execution's status is "stopped". Just before my program exited the execute method, both the job and step executions' status were "started".

So, the one I've describe above is the first issue. The second one is when I do a restart on the job, spring batch will run step 2 again, and then step 3. So if I'm transferring 50GB file, I would prefer for step 2 to not run anymore considering it has finished transferring the 50GB file when I first sent it the stop signal. Is there a way to listen in for 'stop' signal or even catch the JobInterruptedException from within the execute method of the step implementation class?

Thanks again.

[Dave Syer]

The detail certainly helps to clarify what you need. The detection of stop signals is up to the Step implementation. Spring Batch provides a number of Step implementations and the one that people use most is the TaskletStep which looks for the stop signal after each Tasklet execution. If you wrote your own Tasklet and you want it to detect the signal itself then you just look at the StepExecution periodically inside your own code and react to it as you see fit. That should deal with "issue 1" above.

As far as "issue 2" is concerned, there is no way for the framework to know that your stopped step is actually complete unless you tell it so. If you know otherwise, you can manipulate the flow to tell the system not to execute that step on a restart. There are many ways to achieve that, but I think it has to be a business decision.

For instance you could put in an extra step or decision and pass control to it only if the status of the previous execution is STOPPED (<next on="STOPPED" to="..."/>). As soon as control passes out of the stopped step it will be abandoned, so it is not executed again on restart.

You could also use your Tasklet to manipulate the ExecutionContext of the step, so that you drop something in there that you recognise on open() to avoid duplicate processing (assuming your Tasklet is an ItemStream). That would probably be my starting point if I was implementing your use case.

[gerson721]

If you wrote your own Tasklet and you want it to detect the signal itself then you just look at the StepExecution periodically inside your own code and react to it as you see fit. That should deal with "issue 1" above.

Thanks for that Dave. There's actually an issue with your suggested fix to "issue 1". I tried that yesterday before I sent off my second post but that didn't work.

The way I transfer these big files is I have a Tasklet step transfer them one by one. To test the scenario I've described above I use threads and run them side by side, the first one runs the job (and the steps) while the second thread controls the suspension and resumption (through stop and restart methods of the JobOperator) of the job. Inside the execute method of the Tasklet implementation class I've written, I periodically checked to see if the value of StepExecution's status has changed to "Stopped". The status' value remained in "Started" status even until before my application exited the Tasklet's execute method. Only after it exited the execute method did I see the stacktrace for the JobInterruptedException

13:00:57,434 DEBUG [Thread-8] FileCopyTask: Finished up the FileCopyTask at 44 seconds 127 millis
13:00:57,435 DEBUG [Thread-8] FileCopyTask: ************************** exiting execute method - status is: STARTED
13:00:57,484 ERROR [Thread-8] ThreadStepInterruptionPolicy: Step interrupted
13:00:57,487 ERROR [Thread-8] AbstractStep: Encountered an error executing the step
org.springframework.batch.core.JobInterruptedException: Job interrupted status detected.
	at org.springframework.batch.core.step.ThreadStepInterruptionPolicy.checkInterrupted(ThreadStepInterruptionPolicy.java:44)
	at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:365)

You could also use your Tasklet to manipulate the ExecutionContext of the step, so that you drop something in there that you recognise on open() to avoid duplicate processing (assuming your Tasklet is an ItemStream). That would probably be my starting point if I was implementing your use case.

Thanks for the suggestion, I'll probably go with your "write something in the ExecutionContext" approach to avoid rerunning the step that has finished, once "issue 1" has been sorted. Any ideas on how that can be fixed?

The "stopping a job section" http://static.springsource.org/sprin...l#stoppingAJob says that there's no way of catching the "stopped" status from within the developer code. Only when the control is returned to the framework will the application know that the job has been stopped. If this is the case, then I might not really have a way of catching the stop signal from the execute method of my Tasklet?

Thanks again.

[Dave Syer]

I think you are misinterpreting that section of the user guide (so it probably needs an edit). However, if you use the JobOperator to send the signal, then it has to go through the JobRepository, and the only way to detect the signal is to reload the JobExecution from the back end. The SimpleJobRepository actually does this for you, so all you need to do is update the StepExecution in your Tasklet. In your 2-threaded approach though, the launching thread has a reference to the JobExecution, so you can send the signal directly there instead of through the database if you like.