RMI Works on First Run, then Connection Refused

[jones]

Hi,
First, sorry for the double post. I posted it in 'core container' group before remembering there's a special group for RMI. I let it linger for a couple days.. no answer.. here I am.

Strange problem here. I have an RMI service (RmiServiceExporter) and an RMI client (RmiProxyFactoryBean). The service piece is running in a web application. The client fires up in another JVM on the same machine and makes a bunch of calls to the RMI web app service.
The first time the client runs, it works great. I makes several calls to the RMI service and everything functions perfectly, then it shuts down and goes away.
The second time, I fire up the exact same thing (client), I get....

Caused by: org.springframework.remoting.RemoteConnectFailureException: Could not connect to remote service [rmi://localhost:1199/integrationService]; nested exception is java.rmi.ConnectException: Connection refused to host: 192.168.44.198; nested exception is:
        java.net.ConnectException: Connection refused: connect
        at org.springframework.remoting.rmi.RmiClientInterceptorUtils.convertRmiAccessException(RmiClientInterceptorUtils.java:187)
        at org.springframework.remoting.rmi.RmiClientInterceptor.doInvoke(RmiClientInterceptor.java:342)
        at org.springframework.remoting.rmi.RmiClientInterceptor.invoke(RmiClientInterceptor.java:258)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        at $Proxy6.getModelObjLabel(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
        ... 11 more
Caused by: java.rmi.ConnectException: Connection refused to host: 192.168.44.198; nested exception is:
        java.net.ConnectException: Connection refused: connect
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
        at sun.rmi.server.UnicastRef.invoke(Unknown Source)
        at org.springframework.remoting.rmi.RmiInvocationWrapper_Stub.invoke(Unknown Source)
        at org.springframework.remoting.rmi.RmiClientInterceptor.doInvoke(RmiClientInterceptor.java:393)
        at org.springframework.remoting.rmi.RmiClientInterceptor.doInvoke(RmiClientInterceptor.java:339)
        ... 20 more
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at java.net.Socket.<init>(Unknown Source)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(Unknown Source)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(Unknown Source)
        ... 27 more

I have to restart the web app (rmi service) in order to get the client to run successfully again. I'm at a complete loss. It seems to have something to do with the first JVM that connected or the way it was disconnected when it was done... but i'm just grasping at straws.
Any ideas?

Thanks!

[denis.zhdanov]

Try to set refreshStubOnConnectFailure property to true for the RmiProxyFactoryBean.

[jones]

Thank you for the suggestion. Unfortunately, it didn't change anything for me
I'm just not sure how to debug this. Any other ideas?

[denis.zhdanov]

1. Provide complete RmiProxyFactoryBean and RmiServiceExporter bean definitions here;
2. Check if the registry is still up when the failure occurs. Try to perform manual list/lookup if it is. Try to perform the calls from another application;
3. Spring RMI is mostly wrapper around standard RMI. Try to use manually coded RMI for communication and compare results. If the problem reproduces (connection is lost after couple of successful calls) it means that you have a problem with the underlying software (JVM, network protocol stack, OS settings etc);

[jones]

Hi Denis,
Thank you so much for your help. As is usually the case, it was operator error.

Your comments prompted me to start looking behind the scenes in my app beyond just Spring/RMI. It turns out that buried deep in the chain of events that are going on during the process, a singleton was being instantiate (one that wasn't supposed to be) and in effect, firing up another Spring instance with the same RMI configuration on the same port, etc.
So, this would cause contention, or kill one or both listeners on the port and subsequent runs would not connect.
All fixed now. Thanks again!

[denis.zhdanov]

Welcome, sometimes it's necessary just to get someone to talk about the problem and you see the solution