Jun 17th, 2011, 08:04 PM
DefaultMessageListenerContainer recreating Connection during shutdown
My application has a single JNDI based ConnectionFactory, wrapped in a CachingConnectionFactory, used by Camel (which manages the DefaultMessageListenerContainer). The connection gets created during startup correctly, but then when the BeanFactory shuts down, its creating another Connection that it immediately closes.
I tracked this down to an ordering problem between SingleConnectionFactory.destroy and DefaultMessageListenerContainer$AsyncMessageListen erInvoker.
- SingleConnectionFactory.destory() is called by BeanFactory due to being a DisposableBean, causing the connection to be closed and removed from SingleConnectionFactory.
- Camel calls DefaultMessageListenerContainer.stop() from a different thread then the thread that called SingleConnectionFactory.destroy().
- DefaultMessageListenerContainer$AsyncMessageListen erInvoker ends up with an exception because its trying to use the closed Connection Session. DefaultMessageListenerContainer$AsyncMessageListen erInvoker.recoverAfterListenerSetupFailure is called, causing a new Connection to be created by SingleConnectionFactory.
The ordering of these steps is indeterminate since it depends on the order of beans being processed for DisposableBean and thread evaluations (DisposableBean kicks off the Camel shutdown process). Even if I reorder the beans so camel shutdown before CachingConnectionFactory, I can't guarantee the DefaultMessageListenerContainer.stop() is called before SingleConnectionFactory.destroy().
I'm not even sure this is a fixable bug since the best that could happen is SingleConnectionFactory notices its already shutdown and throws an exception, causing log spam until DefaultMessageListenerContainer is shutdown and its AsyncMessageListenerInvoker notices the state change.
Anyone have any thoughts?