OutOfMemory using HibernateCursorItemReader

[cgendreau]

Hi there,

I have a table with ~140 000 rows and each records are relatively big.
So, I want to use a HibernateCursorItemReader to read them in chunk of 500.

My code is:

	<b:job id="copyFromHibernate" job-repository="jobRepository">
		<b:step id="Step-1">
	        <b:tasklet transaction-manager="hibernateTransactionManager">
	            <b:chunk reader="myHibernateReader" writer="myWriter" commit-interval="500"/>
	        </b:tasklet>
	    </b:step>
	</b:job>

	<bean id="myHibernateReader" class="org.springframework.batch.item.database.HibernateCursorItemReader"> 
	    <property name="sessionFactory" ref="sessionFactory" />
	    <property name="useStatelessSession" value="true"/>
	    <property name="fetchSize" value="500"/>
	    <property name="queryString">
			<value>
				<![CDATA[ FROM MyModel]]>
			</value>
	    </property>
	</bean>

With this code, I get an OutOfMemory exception and I can say that the Reader(or Hibernate) is trying to read more than 500 rows because if I add a LIMIT of 500, it works.

I tested with my own reader :

...
	@Override
	public MyModel read() throws Exception, UnexpectedInputException,
			ParseException, NonTransientResourceException {
 		//sessionFactory is not ready in open(...)?
		if(session == null){
			session = sessionFactory.openStatelessSession();
			session.beginTransaction();
			Query criteria = session.createQuery("FROM OccurrenceModel");
			criteria.setFetchSize(500);
			iterator = criteria.scroll(ScrollMode.FORWARD_ONLY);
		}
		
		if(iterator.next()){
			return (MyModel)iterator.get()[0];
		}
		else{
			return null;
		}
	}

With this code, it works. I took a look at the code of HibernateCursorItemReader and it's more or less the same approach.
The database is Postgresql and the TransactionManager is a hibernate4.HibernateTransactionManager.

Any idea what I'm doing wrong with the HibernateCursorItemReader?

[cgendreau]

I have compared the 2 Query objects and the only differences are in the session Object.
transactionCoordinator Object:
-ownershipTaken : false in HibernateCursorItemReader and true in the CustomReader

transactionCoordinator->currentHibernateTransaction Object
-isDriver : false in HibernateCursorItemReader and true in the CustomReader
-managedConnection : null in HibernateCursorItemReader and "Jdbc4Connection" in the CustomReader
-wasInitiallyAutoCommit : false in HibernateCursorItemReader and true in the CustomReader

Do HibernateTransactionManager can be the source of the problem even if I used the same in the 2 readers?

[rishishehrawat]

As per javadocs "Fetch size used internally by Hibernate to limit amount of data fetched from database per round trip".
So it seems like Hibernate tries to load all data in the table in a single round trip when this is not specified causing OOM in application.

[cgendreau]

yes, it looks like this.
I'm now using HibernatePagingItemReader since it's closer to want I'm trying to do.
I have no problem with this reader so far.