concurrently processing messages from jms backed channel

[mshmsh]

I have defined a jms queue backed channel as

<jms:channel id="s3Channel" queue-name="s3FileQueue" connection-factory="connectionFactory" message-driven="true" task-executor="s3Executor" concurrency="5-25"/>

<task:executor id="s3Executor" pool-size="10"/>


<si:service-activator id="s3TransferProcessingJobber"
input-channel="s3Channel"
output-channel="messageTerminusChannel"
ref="mybean"
method="execute"/>


I have message producer that is putting messages in the jms queue: there are over 50 messages in the queue

The problem is that only one message is being processed at one time instead of 5 messages being processed concurrently. I bring up the activemq web console and I can see 5 consumers attached to "s3FileQueue". But I can see only one message being processed.

What am I missing ?

Thanks for your help!

[oleg.zhurakousky]

Are you saying that the second message is not being processed until the processing of the first one is finished?
Try to set acknowledge="auto" attribute

[mshmsh]

That did not work. The messages in this jms backed channel are sill being processing sequentially. There were 20 messages in the queue. Upon starting the integration module, I look at the active mq web console and I see only 5 consumers. Also the number of pending messages stays at 20 even thoug the logger output shows that 1 message is being processed. After this message is completely processed, the number of pending messages is decremented by one,

Still -- only one message is being processed at one time.

This is how my jms channel definition looks:

<jms:channel id="s3Channel" queue-name="s3FileQueue" connection-factory="connectionFactory" acknowledge="auto" message-driven="true" task-executor="s3Executor" concurrency="5-25"/>

[oleg.zhurakousky]

Well, in a way it does make sense, since JMS is enforcing the semantics of the queue - FIFO and ordering. It has to receive an acknowledgment from the first message before it can distribute the second.
Having said that there is another acknowledgment mode acknowledge="dups_ok" which will instruct the session to lazily acknowledge the delivery of messages, however it may result in the delivery of some duplicate messages and/or non-FIFO distribution and lost ordering if JMS fails.

[mshmsh]

Understood. But duplicate messages are unacceptable.

Nevertheless it is possible to use spring jms adapters + activemq and achieve concurrent message processing without duplicates in the normal JMS only scenario.

Is it possible to achieve concurrent message processing with JMS channel adapter without the risk of duplicate messages?

[Mark Fisher]

You might be witnessing the ActiveMQ "prefetch" behavior. They have a very high default setting for that. Try setting the prefetch to 0 and see if that triggers creation of more consumers. Otherwise, the existing consumers might be handling the message load, albeit by prefetching messages from the queue.

[mshmsh]

Thanks for the clarification. It was a configuration parameter on the activemq queue

I whipped up an stand alone poc with an isolated/dedicated activemq just to test the concurrency behavior and ot works as expected. Multiple concurrent consumers are working as expected.

Thanks

[Mark Fisher]

Glad to hear you got it working as expected! Can you elaborate just a bit on the configuration setting you changed?... in case someone else experiences the same issue and searches the forum for answers.

Thanks,
Mark

[mshmsh]

HI Mark

I blew away everything and started from scratch for a quick poc.

Here is my maven project archive. SimplePOCRunnerJunitTest,java (junit file) creates a bunch of messages that are processed concurrently by the consumers.

I hope someone would find this useful.

Thanks!

ps: I am looking forward to the Mannning (MEAP) "Spring Integration in Action "chapters on concurrency.