Connection Pool Exception: Cannot get a connection, pool error Timeout waiting for id

[skaz]

I am running a web application in Production that recently crashed after it was under some stress. I would guess 100-300 people were accessing the site at similar times, which I would expect to work fine.

The logs around the time of the crash are:

org.codehaus.groovy.grails.web.taglib.exceptions.GrailsTagException: Error executing tag <g:render>: Hibernate operation: could not inspect JDBC autocommit mode; uncategorize\
d SQLException for SQL [???]; SQL state [null]; error code [0]; Cannot get a connection, pool error Timeout waiting for idle object; nested exception is org.apache.commons.db\
cp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object at /WEB-INF/grails-app/views/layouts/file.gsp:37
        at gsp_file_gsp$_run_closure2.doCall(file_gsp.groovy:43)
        at gsp_file_gsp$_run_closure2.doCall(file_gsp.groovy)
        at gsp_file_gsp.run(gsp_file_gsp.groovy:48)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at java.lang.Thread.run(Thread.java:662)
Caused by: org.springframework.jdbc.UncategorizedSQLException: Hibernate operation: could not inspect JDBC autocommit mode; uncategorized SQLException for SQL [???]; SQL stat\
e [null]; error code [0]; Cannot get a connection, pool error Timeout waiting for idle object; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot get a co\
nnection, pool error Timeout waiting for idle object
        at User.find(User.groovy:68)
        at User$find.call(Unknown Source)
        at gsp_pps_file_gsp.run(gsp_file_gsp.groovy:22)
        ... 9 more
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object
        at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:114)
        at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
        at $Proxy7.getAutoCommit(Unknown Source)
        ... 12 more
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
        at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1144)
        at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
        ... 14 more

I believe this is directly related to some code I added recently to my Grails project that affect the Connection Pool (although I believe this is not a grails specific problem):

maxActive = 50
maxIdle = 15
minIdle = 5
initialSize = 15
minEvictableIdleTimeMillis = 180000
timeBetweenEvictionRunsMillis = 180000
maxWait = 10000
validationQuery = "/* ping */"

I am using ConnectorJ and a pretty default Grails (1.3.7) configuration.

Also of note: When I

Select processlist

in MySQL I get around 5 connections. Shouldn't this not be possible given my database pooling parameters? They seem to be set right because I have the app-info plugin which confirms the values listed above.
What am I doing wrong? Please help! Thank you.

[Marten Deinum]

In general this is due to wrong (or absent) transaction management, spring needs it to determine when to release resources, if not they hang around until they time out.

[skaz]

Hey Marten,

Thanks for replying. I inherited a Grails project and was under the impression that Transactions were handled automatically. Do you have any pointers as to how I can go about finding where the problems may be? Thanks so much.

[skaz]

I am looking into it, and I don't see anywhere that the code handles transactions in a strange way. We just have services that use "transactional = true", but outside of that nothing with transactions that I can find. Shouldn't Spring handle everything correctly from there? What else would I need to look out for? Thanks.