Thread safety and using GridGain technology with Spring Batch

[fgarsombke]

First of all, great open source product!

I have my infrastructure built out and spring batch works great when I run jobs sequentially.

I have now added GridGain to the mix and am testing its capability to grid my jobs. I can definitely grid the jobs but I am hitting some thread safety issues.

Biggest of all is my input resource:

<bean id="fileResource" class="org.springframework.batch.core.resource.Ste pExecutionResourceProxy">
<property name="filePattern" value="%file.name%"/>
</bean>

This resource object is getting stepped on in each job. It looks like the jobs created by spring batch use the same singleton bean. Similar for my tasklet which I inject the resource into. The tasklet simply removes the resource at the end of the transaction but there is no telling which resource it has a reference to at any given time.

My question is...should I declare these items as non-singletons? What is the best practice if I am going to run these jobs in a truly gridded, parallel environment?

Thanks for a great product.

Franz Garsombke

[dkaminsky]

I'm confused as to what exactly you mean by the resource being removed. Can you elaborate on what you're using the StepExecutionResourceProxy to do and where exactly it's failing?

A potential solution to threading issues with stateless beans is just to use a prototype scope for the beans in question - the proxy is stateless aside from the injected file pattern and resources are immutable, so no big deal creating a new instance for each reference except memory/cpu overhead. You would still be responsible for any resources you create during your step, though.

Alternatively, you could have your classes wrap the proxy using a java ThreadLocal object (call the set/get method of the ThreadLocal from within the public getters/setter) as a holder to ensure that each thread gets its own instance of the proxy and/or delegate resource.

I'm not an official talking head for the project, but I'd say that at this stage in the game, best practice is the same as best practice for concurrency within the rest of Spring and Java at large. Find something that works for you and if you develop useful best practices that others might benefit from, send it back here to the forums and let us know.

[fgarsombke]

Thanks for the response. After digging into some of the scoping threads it sounds like most people are using prototype scoping to solve concurrency issues. I'm sure implementing that will fix my multi-threading problem.

Appreciate your time.

Franz Garsombke.

[Dave Syer]

Best practice for launching jobs is to use a fresh ApplicationContext per job execution - the samples (quartz and task executor launcher) show this with a special purpose JobFactory. Then you can use regular singleton beans, which is better than prototype in acse you want the lifecycle callbacks.