Unable to access context from other threads

[bwawok]

Hi,

Within a tasklet, I need to spawn off many threads. These threads need access to the context, specifically so I can do late binding on readers.

My readers are like so: (10 of these with slightly different tables they access)

	<bean id="myReader"
			class="org.springframework.batch.item.database.JdbcCursorItemReader"
			lazy-init="true"
			init-method="doOpen"
			scope="step"
			autowire="byName" >
		<property name="dataSource" ref="dataSource" />
		<property name="sql">
			<value>
				SELECT *
				FROM my_table
				WHERE column = '#{jobParameters[foobar]}' 
			</value>
		</property>
		<property name="rowMapper">
			<bean
				class="com.foo.bar.rowmapper" />
		</property>
	</bean>

My tasklet is like so:

	<bean id="myTasklet" class="myTaskletClass"
		autowire="byName"
		scope="step"
		lazy-init="true">
	<references to all of the readers I want to read>
	</bean>

What my tasklet is trying to do...

for (ItemReader r : itemReaderList) {
			Thread t = new MyWOrkerThread( r );
			t.start()
}

And within that thread I am trying to do...

		while ( ( record = itemReader.read() ) != null ) {
			doWork( record );
		}

IF I don't need the context, this works fine. I take the scope="step" off the reader, and each thread can do it's own reading.

IF I single thread this, I can use scope="step" on my reader, get the context, and it will work fine.

But if I want to combine both these concepts (scope = step so it replaces parameters, and multithreading) it blows up. The error I get ends up something like...

MyTasklet$MyTHread - org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'lazyBindingProxy.myReader1#sysinit': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'lazyBindingProxy.myReader1#sysinit': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
	at org.springframework.batch.core.scope.util.PlaceholderTargetSource.getTarget(PlaceholderTargetSource.java:185)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)
	at $Proxy0.read(Unknown Source)
	at mycode.....
	at mycode....
Caused by: java.lang.IllegalStateException: No context holder available for step scope
	at org.springframework.batch.core.scope.StepScope.getContext(StepScope.java:197)
	at org.springframework.batch.core.scope.StepScope.get(StepScope.java:139)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
	... 7 more

When I step in the code it is very obvious where it is breaking.. something alone the lines of

StepContext context = StepSynchronizationManager.getContext();

is returning null. The question is, how do I get the thread that I create from within a tasklet to have access to the context?

[litius]

@bwawok,

It seems that what you are trying to do is to run in a "multi-threaded step". Instead of trying to manage threads yourself, is there a particular reason you would not be able to use a "java.util.concurrent.Executor" for it:

<step id="yourStep">
    <tasklet task-executor="taskExecutor">...</tasklet>
</step>

/Anatoly

[bwawok]

So I have tried it with

ThreadPoolTaskExecutor


but it did not help.

Tasklet:

threadPoolTaskExecutor.execute( t );

My applicationContext.xml

	<bean id="threadPoolTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor"
		autowire="byName" />

The error:

2010-10-07 11:49:17,784 ERROR - MyTasklet$MyTHread - org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'lazyBindingProxy.MyReader#sysinit': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'lazyBindingProxy.MyReader#sysinit': Scope 'step' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for step scope
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
	at org.springframework.batch.core.scope.util.PlaceholderTargetSource.getTarget(PlaceholderTargetSource.java:185)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:184)
	at $Proxy0.read(Unknown Source)
	at MyCode...
	at mylasses..MyTasklet$MyTHread.run(MyTasklet.java:86)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalStateException: No context holder available for step scope
	at org.springframework.batch.core.scope.StepScope.getContext(StepScope.java:197)
	at org.springframework.batch.core.scope.StepScope.get(StepScope.java:139)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
	... 10 more

This wasn't exactly your suggestion.. but similar. Using the task-executor on my actual step doesn't make sense, because I don't know how many threads I want. Sometimes my tasklet has 10 things to do and wants 10 threads, other times it has 3 things to do and wants 3 threads.

[litius]

@bwawok,

Using a built-in "multi-threaded step" functionality, the number of threads will generally depend on three things:

1. Your commit interval
2. Number of items to process
3. "throttle-limit" on a "task-executor"

So you would get more threads if it is needed at run time depending on "how many items" you are processing over the "commit interval" up to a "throttle-limit".

I really don't see a good reason to manage threads manually in your use case.

/Anatoly

[bwawok]

Humm okay. I use a task executor like that in quite a few reader - processor - writer steps, but never in a tasklet step.

Any tips on info for how this works? How do I split up my input (in my case a list of ItemReaders), so that each tasklet does 1?

[bwawok]

Oo I think I can use tasklet's return value of RepeatStatus to say if there is more work to do or not. Perfect!