Maximum number of socket connections

[johnedwards]

Am currently working on a project where we are using Spring's lightweight remoting. This application is intended to be able to cope with high numbers of RMI connections being made to the service but we have written some load tests using GroboUtils' MultiThreadedTestRunner and under Windows, when we ramp up to about 50 threads, we currentlyget BeanCreationExceptions being thrown which are wrapping java.rmi.ConnectException :

java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
java.net.ConnectException: Connection refused: connect
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEnd point.java:574)
at sun.rmi.transport.tcp.TCPChannel.createConnection( TCPChannel.java:185)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCP Channel.java:171)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java: 306)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at java.rmi.Naming.lookup(Naming.java:84)

I'm assuming that this is because we're hitting the limit on the number of sockets. I am about to try the same stuff under Linux and, as I understand it, there is a restriction in terms of ulimit (but which can be increased as an environment variable.) I'm curious as to what a reasonable expectation is regarding number of concurrently open sockets. (I appreciate this is a bit of an open-ended question since it probably depends on OS, hardware etc. but I thought we would be able to achieve more than 50 but would definitely be interested to hear other people's experiences.) Also, what strategies have people employed for scaling up RMI? Do people instantiate a number of instances of the service on the same host but with different ports and then round robin on the ports programatically in the client app? Also, I'm wondering if it is possible to load-balance RMI requests in the same way you can load balance HTTP and so round robin on the host as well as the ports but using a physical load balance?

[Alarmnummer]

I`m interrested in the subject but I don`t have an answer to your question. But I do have a few questions.

1) How many concurrent users do you want to support?
2) Are you using a thread per connection? (probably yes)

A thread per connection doesn`t scale well (the overhead of thread-context switching will grow when the number of threads grows). Maybe you can have a look at non blocking io (is part of NIO) to make the system scale better. In combination with non blocking io, you can also have a look at the leader-follower pattern and the reactor/proactor pattern (Patterns of Software Architecture - Patterns for Concurrent and Networked Objects).

[edit]
I haven`t had any problems with 500 concurrent connections (and a thread per connection) *I have written a demo-webcrawler for a customer a few months ago*

[johnedwards]

Thanks for your comments.

> Are you using a thread per connection? (probably yes)

The multithreaded test does use this sort of model and perhaps this not really simulating what we want. Perhaps a more appropriate model for us is that we have small number of client applications making request to this service (let's say number<10 for sake of argument). *But* each of these clients maybe making many (e.g. >100) calls on the remote service. So the model is more like 1 thread per client and each client keeps a reference to the RMI service or pools a number of service instances (if the client is truly multithreaded).