Multi-Threaded transactional problem

[dotbart]

Hello all,

I'm writing an application that connects to emailservers and fetches email from them to eventually put it in a Oracle database.
I'm doing this multi-threaded to compensate for the connection delays.

First, I made a single-threaded prototype, using a Transactional annotated method that synchronises the emailserver with the databaseserver.
This worked just fine.

Now I've tried my hand on multi-threading.
For every different emailserver, it starts a new Thread, in the run() method of that thread, I call the exact function used in the single-threaded prototype.

However, now I get the all known 'LazyInitializationException':
"Exception in thread "Thread-2" org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role"

The full stacktrace is below:

Exception in thread "Thread-2" org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.webmailr.entity.server.MailDirectory.conversation, no session or session was closed
at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:380)
at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:372)
at org.hibernate.collection.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:119)
at org.hibernate.collection.PersistentBag.size(PersistentBag.java:248)
at com.webmailr.persistence.server.MailDirectoryDAOImpl.getLastRecievedMessage(MailDirectoryDAOImpl.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy24.getLastRecievedMessage(Unknown Source)
at com.webmailr.business.mailcheck.fetcher.FolderFetcher.syncConversations(FolderFetcher.java:101)
at com.webmailr.business.mailcheck.fetcher.FolderFetcher.fullSync(FolderFetcher.java:52)
at com.webmailr.business.mailcheck.fetcher.FolderFetcher.fullSyncSubfolders(FolderFetcher.java:79)
at com.webmailr.business.mailcheck.fetcher.FolderFetcher.fullSync(FolderFetcher.java:45)
at com.webmailr.business.mailcheck.fetcher.ServerFetcher.fullSync(ServerFetcher.java:77)
at com.webmailr.business.mailcheck.FetchManagerImpl.startSync(FetchManagerImpl.java:26)
at com.webmailr.business.mailcheck.FetchThread.run(FetchThread.java:37)

I allready tried using the "REQUIRES_NEW" propagation attribute, to create a new connection for every thread, without results.


Does anybody have experience in this field? Or do you know a quick solution to this problem?



Thanks,
Bart

[dotbart]

Really nobody has any idea. It would really help me!

[Alarmnummer]

Do you transfer your entities from one thread to another? If this is the case, I think I know the cause.

Parts of that entity are loaded lazy, this is done by creating some kind of proxy (ieeeuw) with a reference to the session. If you pass this object from one thread to another, you are causing some problems.

1) The session is not threadsafe, so the original thread and the thread you handed your object over to, are reading/writing the same object. So not good.

2) It could be that it takes some time for the thread you handed over your entity to, to use that lazy loaded part. It could be that the initial thread already finished and closed the session. So the other thread wants to load stuff from an already closed session and I think this is the cause of your problem.

What I normally do is:
1) detach from the original session en reattach the entity to the session in the other thread. Detaching makes sure (in theory.. Hibernate had some problems with this in the past) that all references to sessions are removed. If you use optimistic locking there are no problems, but if you don't your system could be subject to lost updates. Adding optimistic locking to your entity solves this problem.
2) or just pass the id (en the version of the entity) to the handover thread, and let that thread do a read again. This approach imho is the cleanest since an entity never leaves a transaction.

ps:
And I would really watch out with oncontroller thread creation and unmanaged threads. I would go for a (ThreadPool)Executor and hook it up from the application context.

[denis.zhdanov]

Really nobody has any idea. It would really help me!

It's really unclear what is happening at your application at the moment. It looks like 'my application works incorrectly, here is the stack trace'. I don't see possibility to provide the solution for such a description

The only advice is to check transaction management because lazy initialization exception most frequently signals about hibernate proxy usage outside of transactional context.

You can try the following then:

1. Set spring logging to the debug level - it will show you when spring infrastructure creates and submits/rollbacks transactions. That may given you a clue to what is happening;
2. Create complete standalone test-case that illustrates the problem and post it here;

[dotbart]

Hi,

thanks for the replies.
I tried your approach of passing the ID and reading the entity again. This seemed like a logical aproach.
The problem still occurs. But it got me thinking.

I use a class I call FetchManager. This controller class contains a few DAO's and starts the whole process of checking a mailserver based on an account.
So when I start a new Thread for an emailserver, this class is used to check it.

However, I create an entity of this class as a Spring bean. When creating a new thread, I use a clone() method on FetchManager, and inject a clone() into the new thread, so it's got the DAO's and so on to work with.

Looking back on it, it's quite likely that the transaction gets cloned aswell, as my first thread ends sooner, that transaction gets closed (as you mentioned) and can't be used further on.

So how do I tackle this issue of creating a new instance of FetchManager without the hassle, or simply make sure it has its own transaction.

REQUIRES_NEW didn't bring a solution.



Thanks,
Bart