Spring batch unexpectedly updating data after data validation exception is thrown

[jonesg]

hello all,
my first post here - I hope I don't cause a commotion

I have a Spring Batch loader which is defined by the following beans:

	<bean id="simpleStep" class="org.springframework.batch.core.step.item.SimpleStepFactoryBean"
		abstract="true">
		<property name="transactionManager" ref="hibernateTransactionManager" />
		<property name="jobRepository" ref="jobRepository" />
		<property name="startLimit" value="1" />
		<property name="commitInterval" value="1" />
	</bean>


	<bean id="skipLimitStep" class="org.springframework.batch.core.step.item.SkipLimitStepFactoryBean"
		parent="simpleStep" abstract="true">
		<property name="skipLimit" value="0" />
	</bean>

   	<bean id="fxRateJob" class="org.springframework.batch.core.job.SimpleJob">  
        <property name="name" value="fxRateJob" />  
   		<property name="jobRepository" ref="jobRepository" />
		<property name="restartable" value="true" />
		<property name="steps">			
				<bean id="fxRateLoad" parent="skipLimitStep">
					<property name="skipLimit" value="25" />
					<property name="transactionAttribute" value="-org.springframework.batch.item.validator.ValidationException" />
					<property name="itemReader" ref="fxRateFile" />
					<property name="itemWriter" ref="hibernateItemWriter" />
				</bean>
		</property>
	</bean>

When a validation exception is thrown, the following behaviour is displayed:

i) If there is no corresponding row in the database for the data row being loaded, then no row will be inserted into the database.
ii) If there is already a row in the database (with valid data - in this case, a FX rate > 0), then even though a exception has been thrown, the invalid FX rate is persisted/updated to the database.

Is the behavour above correct? It seems inconsistent to me - I expect that when a exception is thrown, that no insert or update should take place.

If I drop the skip limit to 1, then I see the behaviour I expect: the existing row in the database is not updated. However, the job exits because the skip limit has been reached - in the real world my skip limit will not be 1.

I've tested this with Spring batch 1.1.3.RELEASE-A and 1.1.4.RELEASE (the no dependecy verisions) and the behaviour seems to be the same for each release.

Any ideas on how I might resolve this?

thanks
G

[lucasward]

That's interesting, I see that you have set the validation exception to not cause rollback, but you haven't set it as skippable. There may be a bug preventing it from causing the job to fail when it should, because if you haven't set the validation exception to be skippable, that's what should happen.

[Dave Syer]

If the exception is not skippable but isn't marked for rollback then it's just a misconfiguration? Or maybe a bug in the writer? I would probably expect in that case that data could be modified persistently, depending on what the reader and writer do.

Make sure you are flushing the Hibernate Session somewhere as well (e.g. in ItemWriter.flush(), or by using the HibernateAwareItemWriter).

[jonesg]

That's interesting, I see that you have set the validation exception to not cause rollback, but you haven't set it as skippable. There may be a bug preventing it from causing the job to fail when it should, because if you haven't set the validation exception to be skippable, that's what should happen.

If the exception is not skippable but isn't marked for rollback then it's just a misconfiguration? Or maybe a bug in the writer? I would probably expect in that case that data could be modified persistently, depending on what the reader and writer do.

Make sure you are flushing the Hibernate Session somewhere as well (e.g. in ItemWriter.flush(), or by using the HibernateAwareItemWriter).

hi, thanks for the replies

Regarding your observations that that my exception isn't marked for roll back - doesn't my configuration define that the validation exception should roll back? As opposed to the "+" prefix, which does specify "no rollback"?

<property name="transactionAttribute" value="-org.springframework.batch.item.validator.ValidationException" />

This is what it states in the sample config here:
http://static.springsource.org/sprin...tml#skipSample

Would I be correct in thinking that if I set this to "+" to not rollback, then I should expect both updates and inserts in my original scenarios (i) and (ii) would be persisted regardless of the exceptions?

If I make my exception skippable:

<property name="skippableExceptionClasses"	value="org.springframework.batch.item.validator.ValidationException"/>

my batch process stops dead after the first validation exception, but when these validation exceptions occur I do want the database transactions to be rolled back, and I only want the process to exit due to validation exceptions once the specified skip limit has been reached.

I am already using the HibernateAwareItemWriter:

	<bean id="hibernateItemWriter"
		class="org.springframework.batch.item.database.HibernateAwareItemWriter">
		<property name="sessionFactory" ref="hibernateSessionFactory" />
		<property name="delegate" ref="hibernateFXRateWriter" />
	</bean>

So, I don't need to worry about flushing the writer?

Thanks
G

[Dave Syer]

You are right about the "-" (but it is probably superfluous, since we'll roll back by default). You are also OK on the flushing if using HibernateAwareItemWriter. You probably should declare the exception as skippable, but that shouldn't be strictly necessary, as all exceptions are skippable by default, so I'm still not sure I understand what's going on. What happens if you remove the transactionAttributes?

Any chance you could produce a test case?