Removing of Step Scope?

[Andy M]

Prototype scope was indeed all I needed to fix my Stax problems, although it wasn't all that straight forward:

1) I was concerned that I was having to push the prototype scope on all parent beans, that included not only the step and job put also various spring Quartz beans that were being injected with by job. Eventual I discovered the prototype “madness” could stop at the MethodInvokingJobDetailFactoryBean I using for my Quartz configuration.
2) When I made my job bean prototype scope, I was having a problem with the jobConfigurationRegistryBeanPostProcessor which somehow was registering my job more than once and throwing a DuplicateJobException. I suspect this may have been a configuration problem which I have now fixed.

It is all working like a dream now, and I have to say, it does seem cleaner after removing all that aop:scoped-proxy stuff.

[Dave Syer]

I feel I should add something here since it was my idea to scrap step scope. There are other concerns to discuss around that decision, but the main one for the purposes of this thread is that prototype scope for readers and writers is not intended to be an adequate replacement for step scope (though it may well work in individual cases). In fact, generally there are many parts of a job that are intrinsically stateful, including the Step implementations themselves potentially, by association with other stateful components. While step scope solved that problem, it only solved it if the user understands it and uses it properly. We need a better solution. In the scope of 1.0 the recommendation is to use a new ApplicationContext for each Job execution. The samples contain two demos of this pattern, re-using the launcher and repository configuration as a parent context for all jobs (see TaskExecutorLauncher and QuartzJobLauncher).

[assaf27]

Using a separate ApplicationContext for each execution as an implicit scope will do the job, but it is a bit constricting in case you're not just running a single job from the command line.

For example if I schedule many jobs in a single JVM, some may get queued and can be waiting for execution for a long while. I now need to keep a reference to both the job and the context so I can close the context when the job is finished (unfortunately a JobListener won't do because it doesn't get called when a job throws an exception).

I thought about advising the job, but I'm not sure how that will work with restarts and when persisting the job. It's a little clumsy.

If I use a step listener to destroy the context then it will be be built for each step in the job which is silly.

I think I liked better the separation between job configuration (which is a singleton) and a job state (which is step scoped).

[Dave Syer]

Good point about the JobListener, but that is too narrow in scope anyway really - the Job has to exist before the JobListeners can get their callbacks. Advising the Job is the wrong scope as well for the same reason, unless you can get a reference to the ApplicationContext in your advice (I can't think of a nice way to do that).

The right level of scope and abstraction is actually JobFactory, and the one that we are using in the samples to demonstrate the ApplicationContext per job execution strategy was missing the close callback for the ApplicationContext. I just fixed it, so you can see a way to do it pretty succinctly in ClassPathXmlApplicationContextJobFactory, if you look in SVN (by checking out the source code or looking in fisheye).

[assaf27]

I checked out the solution you described from SVN. Actually, it looks very much like my first shot at cracking this. However, i soon replaced it with real AOP since when using delegation you loose access to the underlying Job interface (for example in SimpleJob you can call setListeners).
So using an 'After' advice proxy I get the same effect as your solution and more. It still has the problems I mentioned above.

It looks to me like the factory needs to return a different class of objects. Perhaps JobContainer or ConfiguredJob or whatever, and that object should be passed around to other collaborators (for example JobLauncher) which will call its getJob() to get the job and execute it, and dispose() method after execution.

[Dave Syer]

If I understood you I might think about even adding a dispose() hook, even to the Job interface. What do you mean about the delegate? How can an AOP solution have access to more information than the JobFactory?

[assaf27]

What I meant was that the Job can be,for example, a SimpleJob. So its public interface has additional methods (e.g. setListeners). If you wrap it in a delegating Job, you won't be able to access these methods externally. Using AOP (with proxyTargetClass="true") you can access all methods.

[Dave Syer]

OK I get it. I'm not sure I would want to treat the SimpleJob as a public API, except for configuration, so I would call that an abuse of proxyTargetClass=true. But that's a personal opinion - if it works for you, knock yourself out.

[assaf27]

As I said before, I don't like the proxy solution very much.
I would prefer to get a job holder object back from the factory. This object holds both the instantiated Job instance and a reference to the context (or a dispose method).
That object can then be passed around to the other interfaces (e.g. JobLauncher). That means that the other interfaces would have to be aware of JobHolder instead of Job, and they can call dispose after execute is finished.

But this is only my opinion...

[dkaminsky]

Not sure if this is exactly related, but...

What if each JobExecution held an instance of ExecutionAttributes that was automatically persisted just like StepExecution's attributes? That way, stateful job-level attributes can be created by the user using a job or step listener by accessing the job execution, and will be restored if a new execution of the same job instance is created. This would allow inter-step communication, maintain the philosophy of our current paradigm of reinstating state from the database, and remove any need for a "JobContext" (i.e. a non-"won't fix" solution for half of http://jira.springframework.org/browse/BATCH-361 -- the other half [resource disposal] is already addressed by JobListener).