HibernateItemWriter lazy loading problem

[markymiddleton]

I've setup a hibernate job based on the hibernateJob.xml example, with one difference - I'm trying to use lazy loading.

Setup:

Hibernate Entities:
A----*B
The hibernate one-to-many relationship between A and B is defined to be lazy


hibernateItemReader:
<property name="queryString" value="from A" />
<property name="useStatelessSession" value="false"/>

hiberateWriter:
public void write( Object data ) throws Exception
{
A a = (A) data;
a.addB( new B() );
a.addB( new B() );
getSession().saveOrUpdate( a );
}

1st issue:
If the hibernateItemReader does not return any Items (ie. the hibernate query does not return any results) then I get the following exception reported when ever I run the job, although the job appears to return sucessfully:

org.hibernate.SessionException: Session is closed!
at org.hibernate.impl.AbstractSessionImpl.errorIfClos ed(AbstractSessionImpl.java:49)
at org.hibernate.impl.SessionImpl.getBatcher(SessionI mpl.java:260)
at org.hibernate.impl.AbstractScrollableResults.close (AbstractScrollableResults.java:99)
at org.springframework.batch.io.cursor.HibernateCurso rItemReader.close(HibernateCursorItemReader.java:1 12)
at org.springframework.batch.io.cursor.HibernateCurso rItemReader.destroy(HibernateCursorItemReader.java :152)
at org.springframework.beans.factory.support.Disposab leBeanAdapter.destroy(DisposableBeanAdapter.java:1 54):

2nd issue:
If the hibernateItemReader does return results I get the following exception in the hibernateWriter when getSession().saveOrUpdate( a ) is called:

org.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
at org.hibernate.collection.AbstractPersistentCollect ion.setCurrentSession(AbstractPersistentCollection .java:410)
at org.hibernate.event.def.OnUpdateVisitor.processCol lection(OnUpdateVisitor.java:43)

Any ideas ?

[robert.kasanicky]

Does it solve your problem if you turn off lazy loading? Just want to make sure the root cause is correct (no idea what's wrong yet).

[markymiddleton]

If I disable lazy loading, I still get the same exception. Should have checked that!... so its not as I assumed a lazy loading problem.

A bit more info... If I change the hibernateWriter to :

public void write( Object data ) throws Exception
{
A a = (A) getSession().get( A.class , ((A)data).getId() );
a.addB( new B() );
a.addB( new B() );
getSession().saveOrUpdate( a );
}

Then this works fine.

[robert.kasanicky]

I think the the point here is that you have two hibernate sessions: one for reader and another for writer. So:

1st: when there are no items the writer is never called and does not have a chance to open it's session
UPDATE: sorry this is wrong, it's the reader that throws exception on close, not writer
UPDATE2: the error is not specific to your scenario, the original hibernateJob behaves the same way - close is called twice
http://jira.springframework.org/browse/BATCH-343

2nd: the item is associated with the reader's session and you're trying to associate to writer's session

I guess that explains why you're having trouble, now to figure out a solution...

[markymiddleton]

I've found another issue:

Using the hibernateWriter:

public void write( Object data ) throws Exception
{
A a = (A) getSession().get( A.class , ((A)data).getId() );
a.addB( new B() );
a.addB( new B() );
getSession().saveOrUpdate( a );
}


If I attempt to run the hibernateJob on a Weblogic server using a Weblogic datasource and the Weblogic JTA transaction manager I get the following exception after the chunk commit is called:

Caused by: java.sql.SQLException: Result set already closed
at weblogic.jdbc.wrapper.ResultSet.checkResultSet(Res ultSet.java:102)
at weblogic.jdbc.wrapper.ResultSet.preInvocationHandl er(ResultSet.java:58)
at weblogic.jdbc.wrapper.ResultSet_oracle_jdbc_driver _ScrollableResultSet.beforeFirst()V(Unknown Source)
at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:151)
at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)


What seems to be happening is that when the transaction is commited (by the JTA Transaction Manager), the ScrollableResultSet cursor automatically gets closed, and thus when the next item is read I get the above error.

If I attempt to Isolate the HibernateCursorItemReader transaction then everything works fine:


<bean id="transactionManager" class="org.springframework.transaction.jta.WebLogi cJtaTransactionManager">
<property name="transactionManagerName" value="javax.transaction.TransactionManager" />
</bean>

<bean id="propagationNotSupportedTransactionProxy" class="org.springframework.transaction.interceptor .TransactionProxyFactoryBean" abstract="true">
<property name="transactionManager">
<ref bean="transactionManager"/>
</property>
<property name="transactionAttributes">
<props>
<prop key="*">PROPAGATION_NOT_SUPPORTED</prop>
</props>
</property>
</bean>

<bean id="hibernateItemReader" parent="propagationNotSupportedTransactionProxy">
<property name="target">
<bean class="org.springframework.batch.io.cursor.Hiberna teCursorItemReader" scope="step">
<aop:scoped-proxy />
<property name="queryString" value="from A" />
<property name="sessionFactory" ref="sessionFactory" />
<property name="useStatelessSession" value="false"/>
</bean>
</property>
</bean>

Is this the correct way to get around the problem ?

[robert.kasanicky]

Can you please paste the whole stacktrace? The last line of it says the hibernate reader is being reset which indicates some error was encountered before.

[markymiddleton]

- Job: [SimpleJob: [name=hibernateJob]] launched with the following parameters: [{}{}{}]
- select a0_.id as id0_, a0_.attrib_1 as attrib2_0_, a0_.attrib_2 as attrib3_0_ from a a0_
- select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
- select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
- select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
- select hibernate_sequence.nextval from dual
- select hibernate_sequence.nextval from dual
- select hibernate_sequence.nextval from dual
- select hibernate_sequence.nextval from dual
- select hibernate_sequence.nextval from dual
- select hibernate_sequence.nextval from dual
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
- update a set attrib_1=?, attrib_2=? where id=?
- SQL Error: 0, SQLState: null
- Result set already closed
- TransactionSynchronization.afterCompletion threw exception
org.hibernate.exception.GenericJDBCException: exception calling beforeFirst()
at org.hibernate.exception.SQLStateConverter.handledN onSpecificException(SQLStateConverter.java:103)
at org.hibernate.exception.SQLStateConverter.convert( SQLStateConverter.java:91)
at org.hibernate.exception.JDBCExceptionHelper.conver t(JDBCExceptionHelper.java:43)
at org.hibernate.exception.JDBCExceptionHelper.conver t(JDBCExceptionHelper.java:29)
at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:154)
at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)
at org.springframework.batch.item.stream.SimpleStream Manager$3$2.execute(SimpleStreamManager.java:189)
at org.springframework.batch.item.stream.SimpleStream Manager.iterate(SimpleStreamManager.java:212)
at org.springframework.batch.item.stream.SimpleStream Manager.access$100(SimpleStreamManager.java:44)
at org.springframework.batch.item.stream.SimpleStream Manager$3.afterCompletion(SimpleStreamManager.java :186)
at org.springframework.transaction.support.Transactio nSynchronizationUtils.invokeAfterCompletion(Transa ctionSynchronizationUtils.java:133)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.invokeAfterCompletion(Abs tractPlatformTransactionManager.java:951)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.triggerAfterCompletion(Ab stractPlatformTransactionManager.java:926)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.processRollback(AbstractP latformTransactionManager.java:829)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.rollback(AbstractPlatform TransactionManager.java:777)
at org.springframework.batch.item.stream.SimpleStream Manager.rollback(SimpleStreamManager.java:229)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor$1.doInIteration(SimpleStepExecuto r.java:245)
at org.springframework.batch.repeat.support.RepeatTem plate.getNextResult(RepeatTemplate.java:324)
at org.springframework.batch.repeat.support.RepeatTem plate.executeInternal(RepeatTemplate.java:201)
at org.springframework.batch.repeat.support.RepeatTem plate.iterate(RepeatTemplate.java:131)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.execute(SimpleStepExecutor.java:1 86)
at org.springframework.batch.execution.step.simple.Re peatOperationsStep.execute(RepeatOperationsStep.ja va:89)
Caused by: java.sql.SQLException: Result set already closed
at weblogic.jdbc.wrapper.ResultSet.checkResultSet(Res ultSet.java:102)
at weblogic.jdbc.wrapper.ResultSet.preInvocationHandl er(ResultSet.java:58)
at weblogic.jdbc.wrapper.ResultSet_oracle_jdbc_driver _ScrollableResultSet.beforeFirst()V(Unknown Source)
at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:151)
at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)
at org.springframework.batch.item.stream.SimpleStream Manager$3$2.execute(SimpleStreamManager.java:189)
at org.springframework.batch.item.stream.SimpleStream Manager.iterate(SimpleStreamManager.java:212)
at org.springframework.batch.item.stream.SimpleStream Manager.access$100(SimpleStreamManager.java:44)
at org.springframework.batch.item.stream.SimpleStream Manager$3.afterCompletion(SimpleStreamManager.java :186)
at org.springframework.transaction.support.Transactio nSynchronizationUtils.invokeAfterCompletion(Transa ctionSynchronizationUtils.java:133)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.invokeAfterCompletion(Abs tractPlatformTransactionManager.java:951)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.triggerAfterCompletion(Ab stractPlatformTransactionManager.java:926)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.processRollback(AbstractP latformTransactionManager.java:829)
at org.springframework.transaction.support.AbstractPl atformTransactionManager.rollback(AbstractPlatform TransactionManager.java:777)
at org.springframework.batch.item.stream.SimpleStream Manager.rollback(SimpleStreamManager.java:229)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor$1.doInIteration(SimpleStepExecuto r.java:245)
at org.springframework.batch.repeat.support.RepeatTem plate.getNextResult(RepeatTemplate.java:324)
at org.springframework.batch.repeat.support.RepeatTem plate.executeInternal(RepeatTemplate.java:201)
at org.springframework.batch.repeat.support.RepeatTem plate.iterate(RepeatTemplate.java:131)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.execute(SimpleStepExecutor.java:1 86)
at org.springframework.batch.execution.step.simple.Re peatOperationsStep.execute(RepeatOperationsStep.ja va:89)
at org.springframework.batch.execution.job.simple.Sim pleJob.execute(SimpleJob.java:82)
at org.springframework.batch.execution.launch.SimpleJ obLauncher$1.run(SimpleJobLauncher.java:85)
at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
at org.springframework.batch.execution.launch.SimpleJ obLauncher.run(SimpleJobLauncher.java:80)
at org.springframework.batch.execution.bootstrap.supp ort.SimpleExportedJobLauncher.run(SimpleExportedJo bLauncher.java:159)
at org.springframework.batch.execution.bootstrap.supp ort.SimpleExportedJobLauncher.run(SimpleExportedJo bLauncher.java:134)
at jrockit.reflect.VirtualNativeMethodInvoker.invoke( Ljava.lang.Object;[Ljava.lang.ObjectLjava.lang.Object;(Unknown Source)
at java.lang.reflect.Method.invoke(Ljava.lang.Object;[Ljava.lang.Object;I)Ljava.lang.Object;(Unknown Source)
- SQL Error: 0, SQLState: null
- Result set already closed
- SQL Error: 0, SQLState: null
- Result set already closed

[robert.kasanicky]

Frankly it is not clear to me why the cursor is being closed (Lucas?, Dave?), but I see nothing wrong with your solution. The reader does not depend on any transactional magic - it is told by the framework all it needs to know, so things should just work.

Btw. please use code tags when pasting stacktraces

[sm@h]

We wrote 2 components called HibernateInputSource and HibernateBatchingObjectPersister, functionalitywise they are very similar if not equal to HibernateCursorItemReader and HibernateAwareItemWriter. (We are using a snapshot version of spring-batch from last september when those components were not yet in the framework so we were forced to write something ourselves.)

What's more we have struggled for days if not weeks with these very exceptions as well: Illegal attempt to associate a collection with two open sessions, Session closed, Result set already closed.

1) Robert rightly points out it is crucial that you're running both the reader and the writer in the same hibernatesession.

2) However, looking at the sources of HibernateCursorItemReader and HibernateAwareItemWriter this seems impossible to accomplish at the moment. The HibernateCursorItemReader creates its own session through the hibernate API: sessionFactory.openSession(). The reader on the other hand uses the spring API to create another one.

=> Here is how we "solved" this issue:

+ Our HibernateInputSource calls sessionFactory.currentSession() instead

+ We set the hibernate.current_session_context_class attribute to "managed". If you don't do this the session will be closed each time a transaction is committed (everytime the commit-interval configured on the step is reached)

+ We define a bean in the applicationcontext with an init-method in which we set the currentsessioncontext through ManagedSessionContext.bind(getSession()) and a destroy-method in which we remove it through ManagedSessionContext.unbind(sessionFactory)

+ In these 2 methods we also influence the currentsessionmechanism of spring through TransactionSynchronizationManager.bindResource(ses sionFactory, new SessionHolder(getSession())) and TransactionSynchronizationManager.unbindResource(s essionFactory). If you don't do this the transaction settings only apply to spring-batch and not to the HibernateInputSource or the HibernateBatchingObjectPersister. Meaning everything will be commited directly dispite the commitinterval being set to something else.


3) After we did all this the solution became more or less usable, unfortunately there is still 1 thing which we never solved. Due to time constraints we had to work around it by closing and opening the cursor each time the commit interval is reached. Anyhow, the problem is that after a transaction is commited the cursor thinks it is still open, however, you can not read from it anymore because Spring seems to have closed the underlying database connection. This gives the famous "Result set already closed" exception you are now experiencing.

[Dave Syer]

If you use the HibernateCursorItemReader you have to treat the objects you get from it with care because they *do* come from a different session, intentionally (so the cursor can stay open for the duration of the step). By default it uses a StatelessSession, so you explicitly get *detached* objects from it. I thought that was pretty sensible, but some users wanted to do lazy loading with those objects so we provided the useStatelessSession flag as an option. Once you set that to "false" it is your responsibility to deal with the consequences. Can anyone see any way for the framework to help out here?