JTA Transaction w/ Legacy System Results in Exception on Commit

[braddavis]

I am trying to configure a transaction over JMS, Oracle, and DB2 running on iSeries [AS400]. It seems when the commit occurs with the legacy system [DB2], an exception occurs in the transaction.

I am currently using:
Oracle 10.2.0.1
iSeries V5R2 [journaling *is* turned on]
JBoss Messaging 1.0.1

The libraries in use include:
Spring 2.0.2
Hibernate 3.1.1
JT400 5.4
Ojdbc 10.1.0.2.0

The message is received through a Spring DefaultMessageListenerContainer, and passed off to my MessageListener impl. At this point, it is wrapped in the transaction, as I inject the transactionManager into the Spring DefaultMessageListenerContainer, obtained from JBoss's JNDI with name: "TransactionManager".

The hibernate session is also using JTA for the DB2 and Oracle connections, configured into JBoss through data sources, and obtained through the JNDI.

Both data sources are "<xa-datasource>" in their respective "-ds.xml" files. Hibernate is aware that these are JTA data sources by the fact that I set the following Hibernate properties in the LocalSessionFactoryBean:

<prop key="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</prop>
<prop key="hibernate.transaction.manager_lookup_class">org.hibernate.transaction.JBossTransactionManagerLookup</prop>

The exception suggests that the problem may be a timeout occurring on the transaction. The TransactionManager is configured to timeout after 300 seconds [5 minutes]; however, my logs suggest that the transaction only takes 3.5 minutes. A warning precedes the exception, and I imagine is ultimately the root cause of the problem. Has anyone seen this exception occurring on JT400 drivers with XA connections to DB2 on AS400?

Here is the exception that I ultimately receive:

2007-06-14 11:35:16,082 DEBUG [org.springframework.transaction.jta.JtaTransactionManager] Triggering beforeCompletion synchronization
2007-06-14 11:35:16,082 DEBUG [org.springframework.transaction.support.TransactionSynchronizationManager] Removed value [org.springframework.orm.hibernate3.SessionHolder@1f05c08] for key [org.hibernate.impl.SessionFactoryImpl@12a4caf] from thread [DefaultMessageListenerContainer-1]
2007-06-14 11:35:16,082 DEBUG [org.springframework.orm.hibernate3.SessionFactoryUtils] Closing Hibernate Session
2007-06-14 11:35:16,082 DEBUG [org.hibernate.jdbc.ConnectionManager] releasing JDBC connection [ (open PreparedStatements: 0, globally: 0) (open ResultSets: 0, globally: 0)]
2007-06-14 11:35:16,092 DEBUG [org.springframework.transaction.support.TransactionSynchronizationManager] Removed value [org.springframework.jdbc.datasource.ConnectionHolder@8f474f] for key [org.jboss.resource.adapter.jdbc.WrapperDataSource@9578c1] from thread [DefaultMessageListenerContainer-1]
2007-06-14 11:35:16,092 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] Returning JDBC Connection to DataSource
2007-06-14 11:35:16,092 DEBUG [org.springframework.transaction.jta.JtaTransactionManager] Initiating transaction commit
2007-06-14 11:35:16,333 WARN  [org.jboss.tm.TransactionImpl] XAException: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=BRADDAVIS/150, BranchQual=, localId=150] errorCode=XAER_RMFAIL
javax.transaction.xa.XAException
	at com.ibm.as400.access.AS400JDBCXAResource.throwXAException(AS400JDBCXAResource.java:921)
	at com.ibm.as400.access.AS400JDBCXAResource.end(AS400JDBCXAResource.java:353)
	at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:127)
	at org.jboss.tm.TransactionImpl$Resource.endResource(TransactionImpl.java:2143)
	at org.jboss.tm.TransactionImpl$Resource.endResource(TransactionImpl.java:2118)
	at org.jboss.tm.TransactionImpl.endResources(TransactionImpl.java:1462)
	at org.jboss.tm.TransactionImpl.beforePrepare(TransactionImpl.java:1116)
	at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:324)
	at org.jboss.tm.TxManager.commit(TxManager.java:240)
	at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
	at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:787)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:539)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:509)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.springframework.jms.listener.DefaultMessageListenerContainer.executeListener(DefaultMessageListenerContainer.java:275)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:369)
	at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottlingRunnable.run(SimpleAsyncTaskExecutor.java:203)
	at java.lang.Thread.run(Thread.java:595)
2007-06-14 11:35:17,064 DEBUG [org.hibernate.jdbc.ConnectionManager] transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
2007-06-14 11:35:17,064 DEBUG [org.springframework.transaction.jta.JtaTransactionManager] Triggering afterCompletion synchronization
2007-06-14 11:35:17,064 DEBUG [org.springframework.transaction.support.TransactionSynchronizationManager] Clearing transaction synchronization
2007-06-14 11:35:17,074 DEBUG [org.jboss.util.NestedThrowable] org.jboss.util.NestedThrowable.parentTraceEnabled=true
2007-06-14 11:35:17,074 DEBUG [org.jboss.util.NestedThrowable] org.jboss.util.NestedThrowable.nestedTraceEnabled=false
2007-06-14 11:35:17,074 DEBUG [org.jboss.util.NestedThrowable] org.jboss.util.NestedThrowable.detectDuplicateNesting=true
2007-06-14 11:35:17,064 ERROR [org.springframework.jms.listener.DefaultMessageListenerContainer] Setup of JMS message listener invoker failed
org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=BRADDAVIS/150, BranchQual=, localId=150] status=STATUS_NO_TRANSACTION; - nested throwable: (javax.transaction.xa.XAException)
Caused by: 
org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=BRADDAVIS/150, BranchQual=, localId=150] status=STATUS_NO_TRANSACTION; - nested throwable: (javax.transaction.xa.XAException)
	at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:372)
	at org.jboss.tm.TxManager.commit(TxManager.java:240)
	at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
	at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:787)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:539)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:509)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.springframework.jms.listener.DefaultMessageListenerContainer.executeListener(DefaultMessageListenerContainer.java:275)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:369)
	at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottlingRunnable.run(SimpleAsyncTaskExecutor.java:203)
	at java.lang.Thread.run(Thread.java:595)
Caused by: javax.transaction.xa.XAException
	at com.ibm.as400.access.AS400JDBCXAResource.throwXAException(AS400JDBCXAResource.java:921)
	at com.ibm.as400.access.AS400JDBCXAResource.end(AS400JDBCXAResource.java:353)
	at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:127)
	at org.jboss.tm.TransactionImpl$Resource.endResource(TransactionImpl.java:2143)
	at org.jboss.tm.TransactionImpl$Resource.endResource(TransactionImpl.java:2118)
	at org.jboss.tm.TransactionImpl.endResources(TransactionImpl.java:1462)
	at org.jboss.tm.TransactionImpl.beforePrepare(TransactionImpl.java:1116)
	at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:324)
	... 10 more
2007-06-14 11:35:17,084 DEBUG [org.jboss.jms.client.remoting.MessageCallbackHandler] MessageCallbackHandler[-2147483644] closing
2007-06-14 11:35:17,094 DEBUG [org.jboss.jms.server.ServerPeer] ServerPeer [server.0] removing consumer -2147483644 from the cache

Thanks for taking a look at it, I really appreciate it.