Memory Issue

[marcKun]

Hello guys,

Im having this weird problem. We've developed a system that is behaving really awkwardly.
On our local it runs like a precious gem (it runs fine), but when we upload it in a server (my manager told me it's a very powerful linux server),
it has a memory issue. First minute after successfully starting tomcat (5.5), we can test the system, but after sometimes, it hang up.

Im just wondering why this is the case. We have developed 3 projects previously and all are running fine on the same server whenever uploaded.
The only differences I can think of (the only difference apparent to me), is the transactional part of my applicationContext.xml.

As you can see, in my previous projects, when I declaritively make my services transactional, I will only specify the package name, and all
transactional services are stored there. In this project, our lead asked us to group services based on logic, so it turned out a lot of packages and classes.
So when i declaratively make my services transactional, i had a lot <aop:advisor>. Just wanna confirm if this really is the one causing the memory issue?
But it's working fine on our local machine (VISTA). Sorry if the question seems stupid but im running out of idea.

Thanks for any response,
-marckun

[marcKun]

Here's the significant part of my applicationContext.xml

<bean id="placeholderConfig"
        class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="location">
            <value>classpath:init.properties</value>
        </property>
    </bean>

    <!--================= TRANSACTIONAL SETUP ======================-->
    
    <!--================================================================ 
        JOTM CONFIG:
        JOTM is an opensource transaction manager for multiple databases.
     =================================================================-->
    <bean id="jotm" class="org.springframework.transaction.jta.JotmFactoryBean">
        <!-- the transaction timeout -->
        <property name="defaultTimeout" value="300" />
    </bean>

    <!--=================================================================
        JOTM  Transaction Manager.
        A TransactionManager is necessary for suspending and resuming 
        transactions, as this not supported by the UserTransaction interface. 
     ==================================================================-->
    <bean id="jotmTransactionManager" class="org.objectweb.jotm.Current" />
    
    <!--==================================================================
    JTA Transaction Manager
    ===================================================================-->
    <bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
        <property name="userTransaction" ref="jotm" />
        <property name="transactionManager" ref="jotmTransactionManager"/>
    </bean>
    
    <!--==================================================================
     SPRING AOP Transaction advice
     ===================================================================-->
    <tx:advice id="txAdvice" transaction-manager="transactionManager">
        <tx:attributes>
            <!-- all methods starting with get are read-only -->
            <tx:method name="is*" read-only="true"/>
            <tx:method name="get*" read-only="true" />
            <tx:method name="find*" read-only="true" />
            <tx:method name="load*" read-only="true"/>
            <tx:method name="retrieve*" read-only="true"/>
            <!-- other methods will get the default transaction settings -->
            <!--<tx:method name="*" rollback-for="SecurityException"/>
            --><tx:method name="*" rollback-for="Exception"/>
        </tx:attributes>
    </tx:advice>
    
    <aop:config>
        <aop:advisor id="createUserTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.admin.services.CreateUserService.createUser(..))"/>
        <aop:advisor id="changePasswordTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.admin.services.ChangePasswordService.changeUserPassword(..))"/>
        <aop:advisor id="changeUserStatusTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.admin.services.ChangeUserStatusService.changeUserStatus(..))"/>
        <aop:advisor id="readOnlyBaseAdminTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.admin.services.BaseAdminService.*(..))"/>
    
        <aop:advisor id="createCustomerAccountTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.agent.services.CreateCustomerAccountService.createCustomerAccount(..))"/>
        <aop:advisor id="createTrialAccountTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.agent.services.CreateTrialAccountService.createTrialAccount(..))"/>
        <aop:advisor id="readOnlyBaseCreateCustomerTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.agent.services.BaseCustomerService.*(..))"/>
        <aop:advisor id="vipGoldTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.agent.services.VipGolServiceImpl.*(..))" />

        <aop:advisor id="promotionRequestTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.promotion.services.RequestPromotionService.applyPromotion(..))"/>
        <aop:advisor id="promotionApproveTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.promotion.services.VerifyPromotionService.approve(..))"/>
        <aop:advisor id="promotionRejectTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.promotion.services.VerifyPromotionService.reject(..))"/>        
        
        <aop:advisor id="accountInfoRequestTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.accountinfo.services.RequestAccountInfoService.applyAccountRequest(..))"/>
        <aop:advisor id="accountInfoApproveTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.accountinfo.services.VerifyAccountInfoService.approve(..))"/>
        <aop:advisor id="accountInfoRejectTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.accountinfo.services.VerifyAccountInfoService.reject(..))"/>

        <aop:advisor id="depositRequestTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.deposit.services.RequestDepositService.deposit(..))"/>        
        <aop:advisor id="depositApproveTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.deposit.services.VerifyDepositService.approve(..))"/>
        <aop:advisor id="depositRejectTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.deposit.services.VerifyDepositService.reject(..))"/>

        <aop:advisor id="withdrawRequestTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.withdraw.services.RequestWithdrawService.withdraw(..))"/>        
        <aop:advisor id="withdrawApproveTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.withdraw.services.VerifyWithdrawService.approve(..))"/>
        <aop:advisor id="withdrawRejectTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.withdraw.services.VerifyWithdrawService.reject(..))"/>
        
        <aop:advisor id="cancelTransferTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.transfer.services.RemoteTransferService.cancelTransfer(..))"/>
        <aop:advisor id="finishTransferTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.transfer.services.RemoteTransferService.finishTransfer(..))"/>
        
        <aop:advisor id="requestOrderTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.deposit.services.RequestOrderService.applyUpdateOnCredit(..))"/>
        <aop:advisor id="netpayTransaction" advice-ref="txAdvice" pointcut="execution(* com.oas.deposit.services.NetpayService.*(..))"/>
        
    </aop:config> 

Thanks,
-marckun

[Marten Deinum]

First off all are you sure it is memory, isn't it an exhausted connection pool or something else jms?. Second why the load of pointcuts?! Why not simply write 1 pointcut which matches your services.

<aop:config>
  <aop:pointcut id="serviceMethod" expression="execution(* com.oas.*.services.*.*(..))" />
  <aop:advisor id="transactionAdvice" advice-ref="txAdvice" pointcut-ref="serviceMethod" />
</aop:config>

If you need to use a load of advisors you might want to consider using @Transactional instead.

Troubleshooting these kind of issues is kind of tricky, in production in general there is more load, there is more data, there is a complexer topology. So comparing your local environment with production is in general not a good indicator.

[Hetal B]

For such situations it is best to really use one of the performance measuring tool like JProfiler, JProbe and check the root cause of memory leaks.

-Hetal

[marcKun]

@Marten: Thanks for the hint. it sure is messy to look at all those aop:advisor, im relatively new though so forgive my blunder.

@Hetal B: Thanks again, ill check it first thing in the morning, this JProfiler or JProbe, never tried this before.

regards,
-marckun

[marcKun]

First off all are you sure it is memory, isn't it an exhausted connection pool or something else jms?. Second why the load of pointcuts?! Why not simply write 1 pointcut which matches your services.

<aop:config>
  <aop:pointcut id="serviceMethod" expression="execution(* com.oas.*.services.*.*(..))" />
  <aop:advisor id="transactionAdvice" advice-ref="txAdvice" pointcut-ref="serviceMethod" />
</aop:config>

If you need to use a load of advisors you might want to consider using @Transactional instead.

Troubleshooting these kind of issues is kind of tricky, in production in general there is more load, there is more data, there is a complexer topology. So comparing your local environment with production is in general not a good indicator.

@ur first question: as far as our network/system admin is telling me, yes it is. I really have no direct access to the server, just uploading it remotely via vpn and ftp. but ill recheck it for sure. just want to know if my messy aop:advisor config is the culprit or has something to do with it at least, so that i can fix it as early as now. but im having a hard time debugging it since i cant replicate this same issue on my local machine.

@ur second point: i guess you are right about the server having more load. but im just wondering why my past projects with similar configuration (minus this aop:advisor) run pretty well. ill be rechecking the logs and stacktrace again. thanks so much.

regards,
-marckun