LockTimeoutException: Unable to acquire conversation lock after 30 seconds

[wiradikusuma]

Hi guys, I'm experiencing this kind of exception:

ERROR [btpool0-3] BaseXMLFilter.doXmlFilter(157) - Exception in the filter chain
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.webflow.conversation.impl.LockTimeoutException: Unable to acquire conversation lock after 30 seconds
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:583)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:501)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
	at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:154)
	at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:260)
	at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:366)
	at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:493)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
	at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:359)
	at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
	at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
...
Caused by: org.springframework.webflow.conversation.impl.LockTimeoutException: Unable to acquire conversation lock after 30 seconds
	at org.springframework.webflow.conversation.impl.JdkConcurrentConversationLock.lock(JdkConcurrentConversationLock.java:44)
	at org.springframework.webflow.conversation.impl.ContainedConversation.lock(ContainedConversation.java:69)
	at org.springframework.webflow.execution.repository.support.ConversationBackedFlowExecutionLock.lock(ConversationBackedFlowExecutionLock.java:51)
	at org.springframework.webflow.executor.FlowExecutorImpl.resumeExecution(FlowExecutorImpl.java:150)
	at org.springframework.webflow.mvc.servlet.FlowHandlerAdapter.handle(FlowHandlerAdapter.java:173)
	at org.springframework.webflow.mvc.servlet.FlowController.handleRequest(FlowController.java:172)
	at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
...

list.xhtml (list of Persons) -> click "Edit" -> editPerson.xhtml (form containing data from selected Person) -> click "Save" -> back to list. I did that several times before it "hangs" and later on threw the exception.

Unfortunately I can't reproduce the error (tried the same case, didn't work).

Am I doing something wrong?

[andythedestroyer]

What can cause this exception? I can't reproduce it. Last night I received 16 of theses exceptions in a row which eventually caused other errors and led to a server crash.

-Andy

[InverseFalcon]

Which version of SWF are you using?

SWF synchronizes on the conversation to prevent basic concurrency issues. I wasn't aware that there was a timeout when trying to acquire the lock. It seems like that would be problematic, since the lock timeout could occur before request timeout.

[andythedestroyer]

WebFlow uses ReentrantLock ( new in java 5 ) instead of intrinsic locking ( synchronized ) for it's locking. Using ReentrantLock.tryLock method to obtain locks allows you to set a timeout, which is what webflow uses.

I have done some digging and am still trying to find out why we are getting this issue. It looks to me that for some still unknown reason to me we are getting a deadlock on our c3p0 connection pool. This is preventing flows from resuming because no one can get a connection from the pool. Then the FlowExecutor cant resume the flow because another request has the lock waiting for a connection from the connection pool. Http requests keep coming in until too many socket are open causing a org.apache.tomcat.util.net.JIoEndpoint.unknown E Socket accept failed
java.net.SocketException: Too many open files
exception. This brings down the whole server.

Another curious and still unexplained phinomina is just before hte SocketException starts getting thrown, the dm server executes a DELETE request on our web bundle, which of course fails. It seems that the dm server is trying to prevent this one bundle from taking down to whole server, which would be great if it worked. I however have found no documentation of this.

-Andy

[jamesBaker09]

Hi,

I've got similar exception when waiting response from a web service that took too long to respond. Is it possible to change the default 30 seconds lockout time to longer time for spring web flow?

Thanks

-James

[Keith Donald]

It is configurable but admittedly it's not the easiest config setting to get at. The actual setting is on SessionBindingConversationManager, which is used by the webflow-executor tag. Unfortunately, the tag doesn't make this setting configurable. We're aware of this inconvenience at the moment and plan to address it in a future release. Would you mind opening a JIRA requesting why you need this? That always helps and ensures it doesn't get lost.

Keith

[jamesBaker09]

Thanks, Keith.

I've opened a jira issue: http://jira.springframework.org/browse/SWF-1107.
Looking forward to this new feature!


James

[pgozal]

Hi Keith,

I have a problem where users can navigate away from a flow before reaching the end state. If I navigate back to the flow it seems that the persistence context keep acquiring new connection so for example if I set max connection to 10 in my DBCP configuration, after 10 times navigating away and back again to the same flow, we have a deadlock because we reach the connection pool cap.

It's also my question that how WebFlow anticipate the case where users navigate away from a flow before reaching an end state? If we use JPA, will the EntityManager acquired for the flow be closed if user navigate away from the flow? If not when will it be closed?

Thanks before,

Peter

[pgozal]

Hi,

In my case I found out that my service layer retrieves a set of entities. The service layer method was encapsulated in a transaction (using @Transactional) and whenever it is executed, Hibernate connection manager open a jdbc connection and upon method call completion release the connection to the pool. Later on, still in the same request , upon rendering, the view template retrieve a lazy loaded Many-To-One association from these entities to be presented in the view. This causes jdbc connection leak, because the lazy loaded Many-To-One association borrows a jdbc connection but somehow the jdbc connection never released/returned to the pool. Each time I get back to the view state, the lazy loaded Many-To-One association causes another jdbc connection to be opened but never released until the connection pool exhausted to its maximum active connection limit. Finally I got a deadlock.

Cheers,

Peter

[Hedin]

We are also get this exception in our program. The reproducing is simple - if you have executing two (or more) ajax requests and one of it is processed for more than 30 second (this can be in the case of long requests for external system) - others will be timeout with this exception.

As workaround you can use request queue for ajax requests at client (and cancelling by this the first 'A' in ajax). The second way - a rewrite request handling to extract any long requests from WebFlow request cycle to another system.