Connection Refused when connecting through Spring-amqp

[MonkeyKnifeFight]

Hey guys this is a totally bizarre problem but I'm at my wits end trying to debug it so I thought I'd see if anyone has seen this issue. Basically I have connected to my RabbitMQ instance in many cases but always using the Rabbit Java client directly. That has worked fine but now I want to integrate some of this functionality into a Spring app so I'd like to use the Spring-amqp abstractions.

The problem is that for some reason every time I try to connect using the Spring-amqp stuff I get a 'Connection Refused' error. I have used the debugger to inspect what is happening when I try to connect with and without Spring and as far as I can tell there is absolutely no difference. I have drilled all the way down to the socket code in java.net.PlainSocketImpl where there is a call like:

socketConnect(address, port, timeout);

And in both cases that call looks exactly the same but when going through Spring it fails. I'm sure I'm overthinking this and it is something simple. For the record my bean setup looks like:

<bean id="rabbitConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
        <property name="username" value="user"/>
        <property name="password" value="pass"/>
        <property name="virtualHost" value="/vhost" />
        <property name="port" value="5627" />
        <property name="host" value="1.2.3.4" />
    </bean>
    
    <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory">
        <constructor-arg ref="rabbitConnectionFactory"/>
    </bean>

    <bean class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
        <property name="queueName" value="someQueue" />
        <property name="connectionFactory" ref="connectionFactory"/>
        <property name="messageListener" ref="listener" />
    </bean>

I would love any ideas. I'm about to start digging into my outbound network traffic because there is obviously SOMETHING different about the two connection requests.

Chris

[ndpar]

Can you provide the full stacktrace?

[MonkeyKnifeFight]

Yeah I can see how that would be helpful

Full trace is:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer#0' defined in class path resource [META-INF/spring/applicationContext.xml]: Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: java.net.ConnectException: Connection refused
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1401)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:512)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:450)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:290)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:287)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:189)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:557)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:842)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:416)
        at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:45)
        at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:548)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:222)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at com.zub.core.Core.main(Core.java:106)
        at com.zub.locator.IntegrationTest.setUp(IntegrationTest.java:50)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:580)
        at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:398)
        at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:145)
        at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:82)
        at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:167)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:104)
        at org.testng.TestRunner.runWorkers(TestRunner.java:712)
        at org.testng.TestRunner.privateRun(TestRunner.java:582)
        at org.testng.TestRunner.run(TestRunner.java:477)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:324)
        at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:319)
        at org.testng.SuiteRunner.privateRun(SuiteRunner.java:292)
        at org.testng.SuiteRunner.run(SuiteRunner.java:198)
        at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:821)
        at org.testng.TestNG.runSuitesLocally(TestNG.java:788)
        at org.testng.TestNG.run(TestNG.java:708)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
        at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
        at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
        at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Caused by: org.springframework.amqp.AmqpIOException: java.net.ConnectException: Connection refused
        at org.springframework.amqp.rabbit.support.RabbitUtils.convertRabbitAccessException(RabbitUtils.java:118)
        at org.springframework.amqp.rabbit.support.RabbitAccessor.convertRabbitAccessException(RabbitAccessor.java:107)
        at org.springframework.amqp.rabbit.listener.AbstractRabbitListeningContainer.initialize(AbstractRabbitListeningContainer.java:138)
        at org.springframework.amqp.rabbit.listener.AbstractRabbitListeningContainer.afterPropertiesSet(AbstractRabbitListeningContainer.java:100)
        at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.afterPropertiesSet(SimpleMessageListenerContainer.java:89)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1460)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1398)
        ... 50 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
        at java.net.Socket.connect(Socket.java:525)
        at java.net.Socket.connect(Socket.java:475)
        at com.rabbitmq.client.ConnectionFactory.createFrameHandler(ConnectionFactory.java:338)
        at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:374)
        at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:427)
        at org.springframework.amqp.rabbit.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:197)
        at org.springframework.amqp.rabbit.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:143)
        at org.springframework.amqp.rabbit.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:132)
        at org.springframework.amqp.rabbit.support.RabbitAccessor.createConnection(RabbitAccessor.java:88)
        at org.springframework.amqp.rabbit.listener.AbstractRabbitListeningContainer.createSharedConnection(AbstractRabbitListeningContainer.java:322)
        at org.springframework.amqp.rabbit.listener.AbstractRabbitListeningContainer.establishSharedConnection(AbstractRabbitListeningContainer.java:296)
        at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doInitialize(SimpleMessageListenerContainer.java:145)
        at org.springframework.amqp.rabbit.listener.AbstractRabbitListeningContainer.initialize(AbstractRabbitListeningContainer.java:133)
        ... 54 more

[Helena]

I would inspect your ConnectionFactory configuration.

First, if you are using the default virtual host, "/" you don't need to configure it, however say your virtual host name is 'development' then you would want to configure value="development":

<property name="virtualHost" value="/vhost" />

The default rabbit port is what you declare, thus you can remove the port
<property name="port" value="5627" />

The hostname - are you configuring a fully-qualified host such as 'someVm.foo.company.com' ? If so try setting the value to just someVm. I do not think SingleConnectionFactory handles fully-qualified host names, however jinterface accepts it AFAIK
<property name="host" value="1.2.3.4" />

Second, try this configuration instead (or convert it to xml):

@ImportResource("classpath:/path/to/*-context.xml")
public class RabbitTestConfiguration {

    /** from a prop file configured in your rabbit-test-context.xml */
    private @Value("${hostname}") String hostname;

    private @Value("${username}") String username;

    private @Value("${password}") String password;

@Bean
    public SingleConnectionFactory singleConnectionFactory() {
        SingleConnectionFactory connectionFactory = new SingleConnectionFactory(hostname);
        connectionFactory.setUsername(username);
        connectionFactory.setPassword(password);
        return connectionFactory;
    }

Are you connecting to a remote node or one on your machine?

[MonkeyKnifeFight]

Right now I'm trying to connect to a non-default vhost. Are you saying I should leave off the '/'? Like:

<property name="virtualHost" value="vhost" />

For the host I am connecting to a remote node that is running within our network so I was just trying to connect with the IP.

I'll try removing the unnecessary configs and see if any of that helps.

Thanks!
Chris

[MonkeyKnifeFight]

Just to provide an update using the host name instead of the IP seemed to help. I was at least able to connect. Then I was getting an EOFException that seems like it was caused by me removing the '/' from the vhost. Once I put that back in things seem to be working.

I'm still not sure why I can't connect using an IP since I know that works using the RabbitMQ libs directly. But I'm not going to spend too much time right now trying to figure it out. Thanks for the help it might have taken me a while to try that.

Thanks!
Chris

[Helena]

EOFs can be related to the erlang handshake. Check the node itself,and your self node cookie first. There are a few more diagnostics one can do beyond that.

[Tuno]

I do see that 5627 is used for port instead of default 5672.