Transactional rollBack if something goes wrong in the internal database.

Hi,

I splitted the datasource for the Business object from the datasource used by Spring Batch. And my problem is that if I cause a runtime exception while the database job data is inserted (like a DAtaIntegrityViolation changing a column type in the BATCH_STEP_EXECUTION table) then the rollback on the JOB_EXECUTION table is not done... I would expect all the data inserted regarding the job to be removed from the database.

I'm using Hibernate for my domain and I defined a session factory for my object:
org.springframework.orm.hibernate3.annotation.Anno tationSessionFactoryBean
referred by a:
org.springframework.orm.hibernate3.HibernateTransa ctionManager

then I have another transaction manager:
org.springframework.jdbc.datasource.DataSourceTran sactionManager
referring to the batch data source.

I configured this advice:

<bean id="jobRepository"
class="org.springframework.batch.core.repository.s upport.JobRepositoryFactoryBean"
p:databaseType="mysql" p:dataSource-ref="batchDataSource" />

<aop:config>
<aop:advisor
pointcut="execution(* org.springframework.batch.core..*Repository+.*(..) )"
advice-ref="txAdvice" />
</aop:config>

<tx:advice id="txAdvice"
transaction-manager="transactionManager">
<tx:attributes>
<tx:method name="*" propagation="REQUIRED_NEW"
isolation="SERIALIZABLE" read-only="false" />
<tx:method name="*" />
</tx:attributes>
</tx:advice>

as seen in the sample; only changing the method name to * because actually my exception arises when the method:
public void saveOrUpdate(StepExecution stepExecution)
is called in the SimpleJobRepository.

But what happen is that the exception is thrown when trying to insert the step_execution data but the row in the job_execution table remains. I also tried changing REQUIRED_NEW to REQUIRE but without any success. Any glue or suggestion on it? Is it because I'm not using a JtaTransactionManager?

I need to be sure that neither the internal database used by Spring batch doesn't risk to be missed up if a runtime exception arise when persisting the job repository data.



Thanks in advance,
Vicio.

One update...

For sure the previous configuration was wrong in the propagation levels. Because a new transaction was always created..

But also this one:

<tx:advice id="txAdvice"
transaction-manager="batchTransactionManager">
<tx:attributes>
<tx:method name="create*" propagation="REQUIRES_NEW"
isolation="SERIALIZABLE" read-only="false" />
<tx:method name="*"/>
</tx:attributes>
</tx:advice>

imply the creation of always a new transaction by the Spring (2.5.2) AbstractPlatformTransactionManager class.

So I tried this:

<tx:advice id="txAdvice"
transaction-manager="batchTransactionManager">
<tx:attributes>
<tx:method name="create*" propagation="REQUIRES_NEW"
isolation="SERIALIZABLE" read-only="false" />
<tx:method name="*" propagation="SUPPORTS" />
</tx:attributes>
</tx:advice>

with a different level for the normal propagation. Now only one common transaction is created but when I run the batch the row in the BATCH_JOB_EXECUTION table is still not removed.

BTW, I think that probably all this is not such a problem... because the job is marked as failed so if the problem in the spring batch tables (or more likely in the connnection) will be fixed in the next execution, the job will run without problems. Someone agrees? I need to prove we are safe and robust against whatever problems can occurs... and I want to use Spring Batch!!! :-)

If you use multiple datasources you have to use a JtaTransactionManager to keep them in sync.

I'm not really an expert on transaction management... so sorry if the question doesn't make sense... :-)

The tables I want to maintain in sync (the spring batch tables) are in the same datasource, then I suppose managed by the same transaction manager. Do I need JTA if what I want to do is only make possible not to have the job row there.

I also tried using an unique schama, so only one datasoure but the behaviour is the same.

But maybe I'm wrong in one point. I'm supposing that, while executing a job for the first time, if there is a problem inserting the row in the step_execution table then deleting the row from the job table is the correct behaviour. Is this right or it's normal that the job information remains persisted?


Thanks,
Vicio.

Sorry, when you said datasources earlier I assumed you meant more than one. If you're just using one there shouldn't be any need for JTA. I'm not sure I understand your question though, but it might be that I'm reading it wrong. Are you trying to somehow separate the transactions for meta-data (operational transaction) from the 'business transaction'? (i.e. the transaction application logic is applied in) Keep in mind that the framework must keep itself tied to this business transaction, since the ExecutionContext is tied to this data as well. For example, if a FlatFileItemReader is used, it stores the current line it's on in the Executioncontext. If there is a rollback of the business transaction, then the meta-data (i.e. the ExecutionContext) must be rolledback too, so that the line number is correct in case a restart is necessary. Outside of this scenario, such as after the job completes successfully, the status itself should be written in a transaction of it's own. The only operation that has to be treated any differently is the creation of a new JobExecution (as I think you've noticed) This is because if two processes are trying to create a JobExecution for the same JobInstance only one can win.

Hi Lucas,

yes, I'm trying to separate transactions for meta-data from the 'business transactions'. What we thought was to physically separate the operational database from the statistical one, because in the future it could be possible to have other statistical information to be upload in a different database and we would like to just extend our batch project.

BTW, really thanks for your answer and for the example you provided... it was exactly what I was looking for to better understand the transactions flow in Spring Batch and the risks using two datasources without integrating JTA. At this stage we decided to put again all the tables in a unique schema, thus using only one datasource and only one transaction manager; but probably in the future the necessity to split the datasources can arise again... could you provide and example (or a pointer) on how better configuring it with Spring Batch.

Thank you very much!!! You guys are the prove that Open source definitively rocks!!! :-)


Ciao,
Vicio.