JMS DMLC not caching connection when using TX despite cacheLevel = CACHE_CONSUMER?

[tmielke]

Hello,

The DMLC java doc 3.0.5.RELEASE says that cacheLevel can be set differently than CACHE_NONE in case of using transactions:

"This non-caching behavior can be overridden through the "cacheLevel" / "cacheLevelName" property, enforcing caching of the Connection (or also Session and MessageConsumer) even in case of an external transaction manager being involved. "

I am running the Spring JMS config as below [1] and configure the DMLC to use CACHE_CONSUMER and to be transacted, using the Spring JmsTransactionManager.

Using that config [1] I would assume that the JMS connection is getting cached by DMLC.
However while debugging I noticed that the JmsTransactionManager always calls into ConnectionFactory.createConnection() as part of the doBegin() implementation for the transaction.
Similarly the cleanup after a commit closes the connection again.
I do not see any JMS resource caching happening at the DMLC level when using the JmsTransactionManager.
Turning off transactions, correctly caches the connection, consumer and session.

Do I misunderstand the javadoc of DMLC or do I miss any further configuration in order to cache the JMS connection?
I am well aware that a CachingConnectionFactory is recommended when using the JmsTransactionManager in order to pool the connection at that level. Still I would like to understand if that javadoc is incorrect or if I can enable JMS resource caching at the DMLC level when using transactions.

Any feedback welcome.


Thanks,

Torsten Mielke
tmielke.blogspot.com


[1] Spring JMS config used
<bean id="jms" class="org.apache.activemq.camel.component.ActiveM QComponent">
<property name="configuration" ref="jmsConfigAmq" />
</bean>


<bean id="jmsConfigAmq" class="org.apache.activemq.camel.component.ActiveM QConfiguration" >
<property name="connectionFactory" ref="jmsPooledConnectionFactory" />
<property name="transacted" value="true"/>
<property name="transactionManager" ref="jmsTransactionManager" />
<property name="cacheLevelName" value="CACHE_CONSUMER"/>
</bean>

<bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTrans actionManager">
<property name="connectionFactory" ref="jmsPooledConnectionFactory" />
</bean>

<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFacto ry">
<property name="brokerURL" value="tcp://localhost:61617" />
<property name="watchTopicAdvisories" value="false" />
</bean>

<bean id="jmsPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFa ctory" >
<property name="maxConnections" value="3"/>
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>

[Gary Russell]

Please use [ code]...[/code ] tags (no spaces inside) around code and config.

See the javadoc for DMLC.setCacheLevel; when an external transaction manager is used, the cache level defaults to CACHE_NONE instead of CACHE_CONSUMER otherwise.

You may also chose to set sessionTransacted to true, without specifying an external tx manager; in which case the default remains at CACHE_CONSUMER.

[tmielke]

Many thanks for the prompt reply Gary.

I understand that CACHE_NONE is used by default in case of running in a transaction. However the Java doc of DMLC explicitly offers to set a different cache level with an external JMS transaction manager, which I have done in my config snippet above.
Should it not cache the connection when using such config? Or do I misunderstand the docs?

[Gary Russell]

I didn't see a DMLC configuration in your example; hence my comments; now I see you are using Camel.

I am not familiar with the camel wrapper, but I do know that setting the cache level works fine on an vanilla DMLC.

[tmielke]

Hello Gary,

Yes, I am using camel-jms in my example. camel-jms sets up a DMLC instance for receiving msgs from the JMS broker. That DMLC instance uses the Spring JmsTransactionManager to begin a new transaction. And that tx manager calls into ConnectionFactory.createConnection for every transaction!
See a sample stack trace below *1).

At this point there is no Camel code involved yet. Camel only gets involved after a msg was received.
The connection is closed again as part of the resource cleanup after a transaction commit.

This procedure of calling ConnectionFactory.createConnection() Connection.close() is repeated for every new transaction (every new msg).
Needless to say that if the JMS connection isn't cached by DMLC then other resources like consumer and session aren't cached either.

Bottom line is I don't see any JMS resource caching happening at Spring DMLC level despite setting CACHE_CONSUMER.
So it still seems to me the DMLC javadoc is wrong or I am missing some configuration that enables caching in DMLC.
Any idea?

Many thanks,
Torsten

*1)
Daemon Thread [Camel (camelContext) thread #0 - JmsConsumer[adam.test]] (Suspended)
owns: org.apache.activemq.pool.PooledConnectionFactory (id=178)
org.apache.activemq.pool.PooledConnectionFactory.c reateConnection(java.lang.String, java.lang.String) line: 119
org.apache.activemq.pool.PooledConnectionFactory.c reateConnection() line: 94
org.springframework.jms.connection.JmsTransactionM anager.createConnection() line: 280
org.springframework.jms.connection.JmsTransactionM anager.doBegin(java.lang.Object, org.springframework.transaction.TransactionDefinit ion) line: 179
org.springframework.jms.connection.JmsTransactionM anager(org.springframework.transaction.support.Abs tractPlatformTransactionManager).getTransaction(or g.springframework.transaction.TransactionDefinitio n) line: 371
org.apache.camel.component.jms.JmsMessageListenerC ontainer(org.springframework.jms.listener.Abstract PollingMessageListenerContainer).receiveAndExecute (java.lang.Object, javax.jms.Session, javax.jms.MessageConsumer) line: 240
org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.invokeL istener() line: 1058
org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.execute OngoingLoop() line: 1050
org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.run() line: 947
java.util.concurrent.ThreadPoolExecutor$Worker.run Task(java.lang.Runnable) line: 886
java.util.concurrent.ThreadPoolExecutor$Worker.run () line: 908
java.lang.Thread.run() line: 680

[Gary Russell]

I don't know what to tell you. I just ran another test to confirm and this works just fine...

<beans:bean id="lc" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
	<beans:property name="cacheLevelName" value="CACHE_CONSUMER" />
	<beans:property name="transactionManager" ref="transactionManager" />
	<beans:property name="sessionTransacted" value="true" />
	<beans:property name="destination" ref="requestQueue" />
	<beans:property name="connectionFactory" ref="connectionFactory" />
</beans:bean>

You can see in the log the same consumer (and hence connection) used for each poll...

2012-02-28 12:09:55,320 [lc-1] DEBUG: org.springframework.jms.connection.CachingConnectionFactory - Creating cached JMS MessageConsumer for destination [queue://queue.demo]: ActiveMQMessageConsumer { value=ID:myhost-42827-1330448994985-2:1:1:1, started=true }
...
2012-02-28 12:09:56,329 [lc-1] TRACE: org.springframework.jms.listener.DefaultMessageListenerContainer - Consumer [Cached JMS MessageConsumer: ActiveMQMessageConsumer { value=ID:myhost-42827-1330448994985-2:1:1:1, started=true }] of session [Cached JMS Session: ActiveMQSession {id=ID:myhost-42827-1330448994985-2:1:1,started=true}] did not receive a message
...
2012-02-28 12:09:57,345 [lc-1] TRACE: org.springframework.jms.listener.DefaultMessageListenerContainer - Consumer [Cached JMS MessageConsumer: ActiveMQMessageConsumer { value=ID:myhost-42827-1330448994985-2:1:1:1, started=true }] of session [Cached JMS Session: ActiveMQSession {id=ID:myhost-42827-1330448994985-2:1:1,started=true}] did not receive a message
...
etc

Are you sure Camel is propagating the configuration properly?

[tmielke]

hhm, interesting. I took your DMLC definition and loaded it into a testcase here. No Camel involved at all, just Spring DMLC, Spring TX Manager and ActiveMQ PooledConnectionFactory.

I still see the same stack trace at runtime where the JmsTransactionManager calls into the ConnectionFactory.getConnection() method every time it starts a new TX.

The logging output also contains these lines, indicating that a connection was retrieved from the pool.

[2012-02-29 11:10:47,389] org.springframework.jms.connection.JmsTransactionManager     DEBUG Created JMS transaction on Session [PooledSession { ActiveMQSession {id=ID:mac.fritz.box-61680-1330509937568-1:2:1,started=false} }] from Connection [PooledConnection { org.apache.activemq.pool.ConnectionPool@4271c5bc }]
[2012-02-29 11:10:47,391] mework.transaction.support.TransactionSynchronizationManager TRACE Bound value [org.springframework.jms.connection.JmsResourceHolder@4054c9a3] for key [org.apache.activemq.pool.PooledConnectionFactory@4ee3990b] to thread [DMLC-1]
[2012-02-29 11:10:48,394] mework.transaction.support.TransactionSynchronizationManager TRACE Retrieved value [org.springframework.jms.connection.JmsResourceHolder@4054c9a3] for key [org.apache.activemq.pool.PooledConnectionFactory@4ee3990b] bound to thread [DMLC-1]
[2012-02-29 11:10:48,405] org.apache.activemq.ActiveMQSession                          DEBUG ID:mac.fritz.box-61680-1330509937568-1:1:1 Transaction Commit :null
[2012-02-29 11:10:48,405] org.springframework.jms.connection.JmsTransactionManager     DEBUG Initiating transaction commit
[2012-02-29 11:10:48,405] org.springframework.jms.connection.JmsTransactionManager     DEBUG Committing JMS transaction on Session [PooledSession { ActiveMQSession {id=ID:mac.fritz.box-61680-1330509937568-1:2:1,started=false} }]
[2012-02-29 11:10:48,405] org.apache.activemq.ActiveMQSession                          DEBUG ID:mac.fritz.box-61680-1330509937568-1:2:1 Transaction Commit :null
[2012-02-29 11:10:48,406] mework.transaction.support.TransactionSynchronizationManager TRACE Removed value [org.springframework.jms.connection.JmsResourceHolder@4054c9a3] for key [org.apache.activemq.pool.PooledConnectionFactory@4ee3990b] from thread [DMLC-1]

Current stack trace of DMLC thread when starting a new Transaction:

Thread [DMLC-1] (Suspended)	
	owns: org.apache.activemq.pool.PooledConnectionFactory  (id=93)	
	org.apache.activemq.pool.PooledConnectionFactory.createConnection(java.lang.String, java.lang.String) line: 106	
	org.apache.activemq.pool.PooledConnectionFactory.createConnection() line: 94	
	org.springframework.jms.connection.JmsTransactionManager.createConnection() line: 280	
	org.springframework.jms.connection.JmsTransactionManager.doBegin(java.lang.Object, org.springframework.transaction.TransactionDefinition) line: 179	
	org.springframework.jms.connection.JmsTransactionManager(org.springframework.transaction.support.AbstractPlatformTransactionManager).getTransaction(org.springframework.transaction.TransactionDefinition) line: 371	
	org.springframework.jms.listener.DefaultMessageListenerContainer(org.springframework.jms.listener.AbstractPollingMessageListenerContainer).receiveAndExecute(java.lang.Object, javax.jms.Session, javax.jms.MessageConsumer) line: 240	
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener() line: 1058	
	org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop() line: 1050	
	org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run() line: 947	
	java.lang.Thread.run() line: 680

[tmielke]

I changed the test to use Springs CachingConnectionFactory instead of ActiveMQ PooledConnectionFactory. The behavior is the same (see stack trace below {1} which is called for every new transaction).
DMLC is not caching any connection nor session nor consumer when tx are enabled. Because of the JmsTransactionManager it always requests a new connection from the ConnectionFactory. As part of the cleanup after a tx.commit() these JMS resources are all closed again and either returned to the pool or destroyed.

My entire Spring config reads as follows {2}.

{1} stack trace for every new transaction.

Thread [DMLC-1] (Suspended)	
	owns: java.lang.Object  (id=182)	
	org.springframework.jms.connection.CachingConnectionFactory(org.springframework.jms.connection.SingleConnectionFactory).createConnection() line: 227	
	org.springframework.jms.connection.JmsTransactionManager.createConnection() line: 280	
	org.springframework.jms.connection.JmsTransactionManager.doBegin(java.lang.Object, org.springframework.transaction.TransactionDefinition) line: 179	
	org.springframework.jms.connection.JmsTransactionManager(org.springframework.transaction.support.AbstractPlatformTransactionManager).getTransaction(org.springframework.transaction.TransactionDefinition) line: 371	
	org.springframework.jms.listener.DefaultMessageListenerContainer(org.springframework.jms.listener.AbstractPollingMessageListenerContainer).receiveAndExecute(java.lang.Object, javax.jms.Session, javax.jms.MessageConsumer) line: 240	
	org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener() line: 1058	
	org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop() line: 1050	
	org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run() line: 947	
	java.lang.Thread.run() line: 680

{2} Spring config used

 <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory"> 
      <property name="brokerURL" value="failover:(tcp://localhost:61616)" /> 
      <property name="watchTopicAdvisories" value="false" />
    </bean>
  
    <bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
      <property name="connectionFactory" ref="cachingConnectionFactory" />
    </bean>
    
    <bean id="cachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory" >
      <property name="targetConnectionFactory" ref="jmsConnectionFactory"/>
    </bean>

    <bean id="DMLC" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
	  <property name="cacheLevelName" value="CACHE_CONSUMER" />
	  <property name="transactionManager" ref="jmsTransactionManager" />
 	  <property name="sessionTransacted" value="true" />
	  <property name="destinationName" value="requestQueue" />
	  <property name="connectionFactory" ref="cachingConnectionFactory" />
	  <property name="pubSubDomain" value="false" />
	  <property name="messageListener" ref="msgListener" />
    </bean>

   <bean id="msgListener" class="org.apache.activemq.test.MyMessageListener" />

[Gary Russell]

Now I see what you are saying; when a transactionManager is defined; the DMLC delegates to it to get a connection instead of using its internal cache.

This is clearly documented in setTransactionManager; it also advises not to use an external transaction manager unless you really need it, and just set sessionTransacted to true instead.

I'll look into it further but can you just remove the transaction manager for now?

[tmielke]

Thanks a lot Gary,

I can confirm that removing the TX manager and using local tx makes DMLC cache the connection (and consumer and session) just fine. The problem is only with the configured JmsTransactionManager.

Also I understand the doc about setTransactionManager() and that its recommended to not set it when using only local JMS transactions. I actually wasn't aware that you could do transactions in Spring JMS without setting a tx manager. It was a big surprise when you told me. And it seems a good few other people don't know that either.

Although I don't really understand the difference between using local JMS transactions and configuring the JmsTransactionManager (which in the end also uses local JMS transactions). Can you answer this?


So we are almost done here.
Just wonder if you would agree that the javadoc on DMLC is at the least confusing when it says:

"This non-caching behavior can be overridden through the "cacheLevel" / "cacheLevelName" property, enforcing caching of the Connection (or also Session and MessageConsumer) even in case of an external transaction manager being involved."

We have not really managed to get that caching working with the JmsTransactionManager. Not sure if other tx managers would behave differently but according to DMLC source code it does not look like.

Thanks a lot for your help all the way through.
Torsten