Server failing under load using transaction attributes.
Hello,
We are seeing our server (resin) failing under heavy load. CPU reaches 100% and never recovers. I believe that we have isolated the problem to something with Spring attribute autoprxy. When we remove the bean config file, the server scales as expected. The file is below. Also a thread dump after the server locks always contains a get from the attribute cache HashMap. I have attached the dump below.
Any help would be appreciated.
Thanks You
Thread Dump ============
"tcpConnection-6802-19" daemon prio=5 tid=0x3513e818 nid=0xa60 runnable [372ff000..372ffdb0]
at java.util.HashMap.get(HashMap.java:325)
at org.springframework.transaction.interceptor.Abstra ctFallbackTransactionAttributeSource.getTransactio nAttribute(AbstractFallbackTransactionAttributeSou rce.java:77)
at org.springframework.transaction.interceptor.Transa ctionAttributeSourceAdvisor.matches(TransactionAtt ributeSourceAdvisor.java:51)
at org.springframework.aop.support.AopUtils.canApply( AopUtils.java:189)
at org.springframework.aop.support.AopUtils.canApply( AopUtils.java:201)
at org.springframework.aop.framework.autoproxy.Abstra ctAdvisorAutoProxyCreator.findEligibleAdvisors(Abs tractAdvisorAutoProxyCreator.java:83)
at org.springframework.aop.framework.autoproxy.Abstra ctAdvisorAutoProxyCreator.getInterceptorsAndAdviso rsForBean(AbstractAdvisorAutoProxyCreator.java:64)
at org.springframework.aop.framework.autoproxy.Abstra ctAutoProxyCreator.postProcessAfterInitialization( AbstractAutoProxyCreator.java:209)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyBeanPostProcessors AfterInitialization(AbstractAutowireCapableBeanFac tory.java:182)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:285)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:204)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:212)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:136)
at org.springframework.context.support.AbstractApplic ationContext.getBean(AbstractApplicationContext.ja va:426)
at com.opensymphony.xwork.spring.SpringObjectFactory. buildBean(SpringObjectFactory.java:56)
at com.opensymphony.xwork.ObjectFactory.buildAction(O bjectFactory.java:77)
at com.opensymphony.xwork.DefaultActionInvocation.cre ateAction(DefaultActionInvocation.java:199)
at com.opensymphony.xwork.DefaultActionInvocation.ini t(DefaultActionInvocation.java:272)
at com.opensymphony.xwork.DefaultActionInvocation.<in it>(DefaultActionInvocation.java:65)
at com.opensymphony.xwork.DefaultActionInvocation.<in it>(DefaultActionInvocation.java:58)
at com.opensymphony.xwork.DefaultActionProxyFactory.c reateActionInvocation(DefaultActionProxyFactory.ja va:32)
at com.opensymphony.xwork.DefaultActionProxy.prepare( DefaultActionProxy.java:124)
at com.opensymphony.xwork.DefaultActionProxy.<init>(D efaultActionProxy.java:75)
at com.opensymphony.xwork.DefaultActionProxyFactory.c reateActionProxy(DefaultActionProxyFactory.java:45 )
at com.opensymphony.webwork.dispatcher.ServletDispatc her.serviceAction(ServletDispatcher.java:271)
at com.opensymphony.webwork.dispatcher.ServletDispatc her.service(ServletDispatcher.java:243)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:103)
at com.caucho.server.http.FilterChainServlet.doFilter (FilterChainServlet.java:96)
at com.opensymphony.webwork.lifecycle.RequestLifecycl eFilter.doFilter(RequestLifecycleFilter.java:67)
at com.caucho.server.http.FilterChainFilter.doFilter( FilterChainFilter.java:88)
at com.gearworks.servlet.SecurityFilter.doFilter(Secu rityFilter.java:76)
at com.caucho.server.http.FilterChainFilter.doFilter( FilterChainFilter.java:88)
at com.caucho.server.http.Invocation.service(Invocati on.java:315)
at com.caucho.server.http.CacheInvocation.service(Cac heInvocation.java:135)
at com.caucho.server.http.RunnerRequest.handleRequest (RunnerRequest.java:346)
at com.caucho.server.http.RunnerRequest.handleConnect ion(RunnerRequest.java:274)
at com.caucho.server.TcpConnection.run(TcpConnection. java:139)
at java.lang.Thread.run(Thread.java:
attribute bean config file ================
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>
<description>
Generic autoproxy definitions enabling declarative transaction management,
based on Commons Attributes attributes. The application context must define
a single PlatformTransactionManager for autoproxying to work.
This file is generic, and not specific to JPetStore. You can use it unchanged
as an application context definition file for your own applications to get
attribute-driven declarative transaction management.
The power of this approach is more apparent when you have many transactional
business objects, not just one as in this simple application.
Add more Advisor definitions if you want, for additional declarative services.
</description>
<!--
This bean is a postprocessor that will automatically apply relevant advisors
to any bean in child factories.
-->
<bean id="autoproxy"
class="org.springframework.aop.framework.autoproxy .DefaultAdvisorAutoProxyCreator">
<property name="proxyTargetClass"><value>true</value></property>
</bean>
<!--
Commons Attributes Attributes implementation. Replace with another
implementation of org.springframework.metadata.Attributes to source
attributes from a different source.
-->
<bean id="attributes"
class="org.springframework.metadata.commons.Common sAttributes"/>
<bean id="transactionAttributeSource"
class="org.springframework.transaction.interceptor .AttributesTransactionAttributeSource">
<constructor-arg>
<ref bean="attributes"/>
</constructor-arg>
</bean>
<bean id="transactionInterceptor"
class="org.springframework.transaction.interceptor .TransactionInterceptor">
<property name="transactionManager">
<ref bean="transactionManager"/>
</property>
<property name="transactionAttributeSource">
<ref bean="transactionAttributeSource"/>
</property>
</bean>
<!--
AOP advisor that will provide declarative transaction management
based on attributes.
It's possible to add arbitrary custom Advisor implementations as
well, and they will also be evaluated and applied automatically.
-->
<bean id="transactionAdvisor"
class="org.springframework.transaction.interceptor .TransactionAttributeSourceAdvisor">
<constructor-arg>
<ref bean="transactionInterceptor"/>
</constructor-arg>
</bean>
</beans>