Spring auto closed with c3p0 exceptions

[cesia]

Hi,

I'm using tomcat6 and spring2.5.6SEC01.

After a period of stable running, spring is auto closed after some c3p0 exceptions, I haven't found the reason. There is the exception below:

[Thread-15] 2010 Sep 04 21:18:37 INFO  org.springframework.web.context.support.XmlWebApplicationContext:815 - Closing org.springframework.web.context.support.XmlWebApplicationContext@2b5356d5: display name [Root WebApplicationContext]; startup date [Sat Sep 04 17:18:37 CST 2010]; root of context hierarchy
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.quartz.core.QuartzScheduler:470 - Scheduler org.springframework.scheduling.quartz.SchedulerFactoryBean#0_$_NON_CLUSTERED paused.
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.springframework.beans.factory.support.DefaultListableBeanFactory:421 - Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4b9a0203: defining beans [propertyConfigurer,messageSource,dataSourceXdj,transactionManagerXdj,baseTxProxyXdj,initBean,jdbcTemplate,mailLoader,gmDao,commonDao,roleDao,itemDao,dishDao,restDao,roomDao,decorationDao,jobDao,chatDao,packDao,guildDao,techDao,kitchenDao,roleSpecialItemDao,friendDao,mailDao,taskDao,userDao,couponDao,shopItemDao,eatItemDao,rewardDao,strategyDao,fcmDao,questionDao,rankDao,trophyDao,marketDao,eventDao,statDao,roleActDao,staffDao,appointmentDao,intimacyActDao,petDao,visitDao,restMoneyDao,carDao,cacheManager,defaultUseItemStrategy,useCouponItemStrategy,useFitmentItemStrategy,useSuperStaffItemStrategy,useVipItemStrategy,usePetItemStrategy,useCarItemStrategy,equipPackTemplate,specialItemPackTemplate,dishPackTemplate,itemPackTemplate,kitchenFitmentTemplate,roomFitmentTemplate,packManager,mailManager,fitmentManager,roleManager,friendManager,broadcastManager,vipManager,restManager,staffManager,petManager,carManager,dishManager,roleActManager,spManager,payManager,roleExecutor,teamExecutor,gotoExecutor,restExecutor,eatExecutor,fitmentExecutor,sessionClosedExecutor,dishExecutor,chatExecutor,friendExecutor,workExecutor,packExecutor,techExecutor,cookExecutor,couponExecutor,npcExecutor,dealExecutor,taskExecutor,shopExecutor,eatGameExecutor,mapExecutor,lotteryExecutor,exchangeExecutor,strategyExecutor,commonExecutor,mailExecutor,statExecutor,questionExecutor,rankExecutor,logExecutor,gmExecutor,enchantExecutor,trophyExecutor,marketExecutor,eventExecutor,staffExecutor,appointmentExecutor,pingExecutor,gameExecutor,petExecutor,visitExecutor,carExecutor,roleProtocol,mapProtocol,restProtocol,decorationProtocol,dishProtocol,roomProtocol,jobProtocol,packProtocol,itemProtocol,techProtocol,kitchenProtocol,couponProtocol,userProtocol,taskProtocol,eatGameProtocol,strategyProtocol,mailProtocol,statProtocol,questionProtocol,shopProtocol,rankProtocol,marketProtocol,gmProtocol,appointmentProtocol,intimacyActProtocol,petProtocol,visitProtocol,carProtocol,rewardProtocol,abstractJobDetail,abstractSimpleTrigger,abstractCronTrigger,restNpcMovieHandler,restNpcMovieJobDetail,restNpcMovieSimpleTrigger,evenRestHandler,evenRestJobDetail,evenRestTrigger,techHandler,techJobDetail,techTrigger,restLevelupHandler,restLevelupJobDetail,restLevelupTrigger,taskHandler,taskJobDetail,taskTrigger,weatherHandler,weatherJobDetail,weatherTrigger,clearHandler,clearDetail,clearTrigger,clearTWHandler,clearTWDetail,clearTWTrigger,onlineCountHandler,onlineCountDetail,onlineCountTrigger,rankHandler,rankDetail,rankTrigger,restWorthRankHandler,restWorthRankJobDetail,restWorthRankTrigger,kitchenEnvRankHandler,kitchenEnvRankJobDetail,kitchenEnvRankTrigger,eventHandler,eventJobDetail,eventTrigger,eventResultHandler,eventResultDetail,eventResultTrigger,couponItemHandler,couponItemDetail,couponItemTrigger,staffHandler,staffDetail,staffTrigger,trainingClearHandler,trainingClearDetail,trainingClearTrigger,godHandler,godDetail,godTrigger,kratosHandler,kratosDetail,kratosTrigger,org.springframework.scheduling.quartz.SchedulerFactoryBean#0,gmService,commonService,roleService,dishService,restService,roleEatService,roomService,chatService,decorationService,roleJobService,packService,kitchenService,techService,taskService,npcService,dealService,shopItemService,eatGameService,itemService,rewardService,strategyService,fcmService,friendService,mailService,statService,questionService,couponService,rankService,userService,trophyService,marketService,eventService,vipService,staffService,appointmentService,intimacyActService,exchangeService,petService,visitService,restMoneyService,carService,org.springframework.beans.factory.config.CustomEditorConfigurer#0,executorFilterThreadPoolFactory,executorFilter,encoder,decoder,codecFilter,loggingFilter,filterChainBuilder,protocolHandler,ioAcceptor843,executorMap,gameHandler,chatHandler,ioAcceptor19365,ioAcceptor19124]; root of factory hierarchy
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.springframework.scheduling.quartz.SchedulerFactoryBean:728 - Shutting down Quartz Scheduler
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.quartz.core.QuartzScheduler:542 - Scheduler org.springframework.scheduling.quartz.SchedulerFactoryBean#0_$_NON_CLUSTERED shutting down.
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.quartz.core.QuartzScheduler:470 - Scheduler org.springframework.scheduling.quartz.SchedulerFactoryBean#0_$_NON_CLUSTERED paused.
[Thread-15] 2010 Sep 04 21:18:37 INFO  org.quartz.core.QuartzScheduler:592 - Scheduler org.springframework.scheduling.quartz.SchedulerFactoryBean#0_$_NON_CLUSTERED shutdown complete.
[pool-2-thread-14] 2010 Sep 04 21:18:37 WARN  com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool:640 - An Exception occurred while trying to check a PooledConection into a ResourcePool.
com.mchange.v2.resourcepool.ResourcePoolException: ResourcePool [BROKEN!]: Tried to check-in a foreign resource!
	at com.mchange.v2.resourcepool.BasicResourcePool.checkinResource(BasicResourcePool.java:657)
	at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$ConnectionEventListenerImpl.doCheckinResource(C3P0PooledConnectionPool.java:636)
	at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$ConnectionEventListenerImpl.connectionClosed(C3P0PooledConnectionPool.java:630)
	at com.mchange.v2.c3p0.util.ConnectionEventSupport.fireConnectionClosed(ConnectionEventSupport.java:55)
	at com.mchange.v2.c3p0.impl.NewPooledConnection.fireConnectionClosed(NewPooledConnection.java:510)
	at com.mchange.v2.c3p0.impl.NewPooledConnection.markClosedProxyConnection(NewPooledConnection.java:381)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.close(NewProxyConnection.java:1246)
	at org.springframework.jdbc.datasource.DataSourceUtils.doReleaseConnection(DataSourceUtils.java:313)
	at org.springframework.jdbc.datasource.DataSourceUtils.releaseConnection(DataSourceUtils.java:274)
	at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCleanupAfterCompletion(DataSourceTransactionManager.java:316)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.cleanupAfterCompletion(AbstractPlatformTransactionManager.java:989)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:782)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy18.doUpdateLogout(Unknown Source)
	at com.deity.xdj.executor.impl.SessionClosedExecutor.executor(SessionClosedExecutor.java:236)
	at com.deity.xdj.handler.GameHandler.sessionClosed(GameHandler.java:363)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.sessionClosed(DefaultIoFilterChain.java:701)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:378)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$900(DefaultIoFilterChain.java:49)
	at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionClosed(DefaultIoFilterChain.java:817)
	at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:87)
	at org.apache.mina.core.session.IoEvent.run(IoEvent.java:64)
	at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:551)
	at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:543)
	at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:487)
	at java.lang.Thread.run(Thread.java:619)

...a lot of same exceptions following.

Please help to give some advice, thank u.

[dr_pompeii]

Hello

After a period of stable running

Is constant such period?

[pool-2-thread-14] 2010 Sep 04 21:18:37 WARN
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool:
640 - An Exception occurred while trying to check a PooledConection into a ResourcePool.
com.mchange.v2.resourcepool.ResourcePoolException:
ResourcePool [BROKEN!]: Tried to check-in a foreign resource!
at com.mchange.v2.resourcepool.BasicResourcePool.chec kinResource(BasicResourcePool.java:657)

1) Can you post your C3P0 configuration?
2) What Framework about persistence you are using?
3) Which DB you are using?
4) In which place of your code could be arising such error? Can you posted?

I see in your error stack the follow

at $Proxy18.doUpdateLogout(Unknown Source)
at com.deity.xdj.executor.impl.SessionClosedExecutor. executor(SessionClosedExecutor.java:236)
at com.deity.xdj.handler.GameHandler.sessionClosed(GameHandler.java:363)

[cesia]

Hello


Is constant such period?



1) Can you post your C3P0 configuration?
2) What Framework about persistence you are using?
3) Which DB you are using?
4) In which place of your code could be arising such error? Can you posted?

I see in your error stack the follow

Thank you for the quick response.

First, such period is not a constant.

Here is my environment:
1)c3p0 configuration

<bean id="dataSourceXdj" class="com.mchange.v2.c3p0.ComboPooledDataSource"
		destroy-method="close">
		<property name="driverClass" value="${jdbc.driverClass}"/>
		<property name="jdbcUrl" value="${jdbc.url}"/>
		<property name="user" value="${jdbc.username}"/>
		<property name="password" value="${jdbc.password}"/>
		
		<property name="maxStatements">
			<value>0</value>
		</property>
		<property name="maxIdleTime">
			<value>1800</value>
		</property>
		<property name="initialPoolSize">
			<value>100</value>
		</property>
		<property name="minPoolSize">
			<value>100</value>
		</property>
		<property name="maxPoolSize">
			<value>300</value>
		</property>
		
		<property name="acquireRetryAttempts">
			<value>0</value>
		</property>
	</bean>

2) no persistence Framework, just Spring JdbcDaoSupport.

3) Oracle10.2.0.4

4) It happens in methods cooperate with db such as the following:

	@Override
	public int updateLogoutInfo(long roleId, Timestamp logout, int hp,
			int appetite, long onlineTime) {
		String sql = "update tb_role set last_logout=?,hp=?,appetite=?,total_online_time=total_online_time+?,this_online_time=? where role_id=?";
		Object[] params = new Object[] { logout, hp, appetite, onlineTime,
				onlineTime, roleId };
		return updateData(sql, params);
	}

I suppose this situation may happen when the db connection pool was exhausted, but I can't confirm because there are no server's cpu or memory logs when Spring auto closed.


More infomation:
Oracle memory: 2G
Java application memory: 3.2G
Server Total memory: 8G

[dr_pompeii]

Hello Cesia

Just some observations

1)

<property name="maxStatements">
	<value>0</value>
</property>

zero?

2)

no persistence Framework, just Spring JdbcDaoSupport.

According with the link is very closely related with JdbcTemplate

because saids

Requires a DataSource to be set, providing a JdbcTemplate based on it to subclasses through the getJdbcTemplate() method.

Therefore in your code where is such template?

What does return updateData(sql, params);? since updateData is not a method in JdbcDaoSupport

Be sure to be using the template to let Spring open and close the resources by you.

[cesia]

Hello Cesia

Just some observations

1)

<property name="maxStatements">
	<value>0</value>
</property>

zero?

2) According with the link is very closely related with JdbcTemplate

because saids



Therefore in your code where is such template?

What does return updateData(sql, params);? since updateData is not a method in JdbcDaoSupport

Be sure to be using the template to let Spring open and close the resources by you.

Thank you for the reply.

Yes, I use JdbcTemplate to operate the database operation.
I admit there is few conditions to identify what's wrong. The server is running well now. Maybe a small bug in my code triggered the runtime error. I must look for it. Next time I will give more logs.