Scheduling and Predecessors

[sky33]

I have just started delving into Spring Batch(nice work!) and I am wondering about handling non-sequential scheduling of tasks. For example, say I have a couple of related jobs, where job B depends on job A to be complete before it can begin processing. Both job A and B are triggered via an external message that can come in at any time. What would you recommend for scheduling based on completion of predecessor jobs/tasks?

[Dave Syer]

I think what I would do is set up two jobs, one that is triggered by message A and one by message B. Job A would have a second step that could complete task B if the trigger was available, and if not then set up a trigger for Job B (e.g. just a record in a database). Job B would execute task B if the trigger from A is there, otherwise set another trigger to tell Job A to run task B next time it fires. Does that work?

This isn't an area we've thought much about yet, but there might be some framework features in there for the future. Let us know what you come up with or if you need anything.

[dkaminsky]

You have several choices here...

1) use a scheduler that supports complex dependency trees -- in this case the scheduler should handle this for you

2) have your scheduler call another program (a dispatcher) to launch your batch jobs, and let this program handle this logic - an example of such a program would be a shell script or scriptable tool such as Ant.

3) Have one job send the message to kick of the next, as suggested earlier.

[sky33]

Thanks for your replies. I do like the triggering option. Makes me wonder if you could then abstract out a trigger so you could trigger by inserting into a table, by JMS message, etc. For the meta data that is managed by the JobRepository, I suppose I could query it to see if specific JobInstance had completed.

As far as utilizing schedulers, would anyone recommend using Quartz to handle more complex scheduling?

[lucasward]

Quartz is fairly handy for triggering, but it doesn't really have any dependency management between jobs.

[sky33]

It also looks like you could utilize job listeners to determine when a job is complete and then potentially queue up another job/process. I think I will try out a few different ways, and see what sticks.