Messages not waiting in queue

[eazynow]

Hi,
I'm writing a client that publishes messages to a topicExchange, and another clients that bind the exchange to their own queue and receive relevant messages fine. (these will be one client soon but for simplicity I'm keeping them separate for the moment)

The problem I have is when the receiving client is not listening - i.e. it hasn't got a SimpleMessageListenerContainer started, or it's not running. I'd expect the queue to fill, as the queue and binding still exist (the queue is durable)

The queue and binding is as follows:

public TopicExchange exch() {
TopicExchange ex = new TopicExchange(exchangeName);
ex.setDurable(true);
return ex;
}
public Queue rcvQueue() {
Queue recQueue = new Queue(this.recQueueName);
recQueue.setAutoDelete(false);
recQueue.setDurable(true);
return recQueue;
}

Binding is done in code:

Binding bind = BindingBuilder.from(receiveQueue).to(exchange).wit h("a.#");
admin.declareBinding(bind);

If I send a bunch of messages, running the following shows my queue but no messages:

rabbitmqctl list_queues name messages_ready messages_unacknowledged messages consumers

Listing queues ...
myTAsyncQueue 0 0 0 0

My bindings:

Listing bindings ...
exchange myTAsyncQueue queue myTAsyncQueue []
myTExchange exchange myTAsyncQueue queue a.# []
...done.

I've got setRequireAck, setMandatoryPublish and setImmediatePublish all to true on the RabbitTemplate and I'm sending using RabbitTemplate:

template.convertAndSend(routingKey, message);

As soon as I start my client again, it picks up further messages but all the ones in between are gone.

I would really appreciate a point in the right direction.

Thanks

[Mark Fisher]

I think what you are seeing is to be expected. The key point is that you are using 'mandatory' and 'immediate' values of true. When 'mandatory' is true, it will require that a Queue is bound to the Exchange at the time that you publish. If 'immediate' is true, it will further require that a Consumer is actively registered on a bound Queue. I think that is the issue ('immediate' being true). If you do want the Messages to be enqueued when there is no active Consumer registered, then set 'immediate' to false.

[eazynow]

Thanks Mark - that did it!

When I saw immediate publish I was assuming that it was talking about when to send the message from the app to the exchange. But as you rightly said, changing that sorted it.

Thanks again

[avyaktha]

I think what you are seeing is to be expected. The key point is that you are using 'mandatory' and 'immediate' values of true. When 'mandatory' is true, it will require that a Queue is bound to the Exchange at the time that you publish. If 'immediate' is true, it will further require that a Consumer is actively registered on a bound Queue. I think that is the issue ('immediate' being true). If you do want the Messages to be enqueued when there is no active Consumer registered, then set 'immediate' to false.

What does it mean to have a consumer actively registered? does it mean that the consumer has a active connection open ? or does it mean that the consumer has an active channel open?

there might be cases where the consumer has a connection open but no channels, in which case it isn't consuming.

[Dave Syer]

The spec isn't very clear about this and a channel is a Rabbit implementation detail. However it is easy to explain in Rabbit terms: a consumer is active if it has sent basicConsume() on a channel and not sent basicCancel(), and the channel is still open.

N.B. immediate and mandatory flags are not very useful to most application messaging patterns, so we removed them from Spring AMQP (https://jira.springsource.org/browse/AMQP-76). The use cases that they support might be added back later if there is demand, but they are not very common, so it's probably better to treat them as artifacts of the spec (being written by a committee with a bunch of different concerns).

[avyaktha]

In my case this would be useful.I have a message queue setup with a constant load of 700 QPS. The messages arent business critical we can afford to loose messages.

We noticed that the consumer wasn't able to catch up with the producer, so often we saw the queue running out of memory and closing the channels of the consumer. So we had to clean up and restart every time.

what would be ideal is the producer to stop putting stuff into the queue when they see there are no active consumers. Ideal case would be to increase the number of consumers? use a non durable queue? . what the best way to deal with this issue?

[Dave Syer]

Messages are simply dropped by the broker if there are no queues bound to matching routing patterns. That sounds like it meets your requirement better than an exception if it can't be delivered? It depends on the detail of your use case, I suppose, but one very common pattern is to use anonymous, non-durable, exclusive queues on the consumers and then messages only reach the consumers when they are actively listening.