Process multiple messages at once

[Miko95]

Hi,

Is it possible to get more than one message at once from a message channel? If yes, which kind of channel provides this feature? Publish/subscribe or queue?

Thanks,
Mickael

[Cleric]

Hello

Let's start from sample:

<task:executor id="taskExecutor" pool-size="5"/>

<channel id="queueChannel">
	<queue/>
</channel>

<channel id="executorChannel">
	<dispatcher task-executor="taskExecutor"/>
</channel>

<bridge input-channel="queue" output-channel="executorChannel">
	<poller fixed-rate="1000"/>
</bridge>

<service-activator input-channel="executorChannel"/>

1. Your messages are received into queueChannel and stored in the real Queue object! ;-)
2. <bridge> polls this channel via PollingTask every sec. But in one pass bridge polls messages from queue until it receives null: one-by-one and handles each of them immediately into output-channel
3. But the last one is an Executor Channel (http://static.springsource.org/sprin...l/ch03s02.html) with dispatcher on ThreadPoolTaskExecutor
4. In the end some <service-activator> is subscribed to "executorChannel".
5. And thanks for "taskExecutor" your messages are processed in parallel.

Is it what are you looking for?

If not: to read from channel more than one message (Collection) will be some anti-pattern for Messaging Architecture. ;-)
However you can use <aggregator> (http://static.springsource.org/sprin...html/ch11.html) to get in the end a Collection of you Messages' payalods.

Cheers,
Artem Bilan

[Miko95]

Hi Artem,

First, thanks for helping me.

5. And thanks for "taskExecutor" your messages are processed in parallel.

When I receive a message, I need to execute an INSERT on a certain database. So, in the case there are many messages in the queue, I thought about receiving N messages at once, so I can use the batch functionality of JdbcTemplate.

I understand from your solution that it's not the case, am I right (since my messages will be processed in parallel)? What I want is more "My groups of N messages will be processed in parallel".

However you can use <aggregator> (http://static.springsource.org/sprin...html/ch11.html) to get in the end a Collection of you Messages' payalods.

May you provide an example, please?
I would like to process N messages at once (or less if there are not enough messages in the channel). I have no idea of which Correlation Strategy to use. Which correlation key should I use for a given message?

Thanks

[Cleric]

Well.

What I see here:

<aggregator correlation-strategy-expression="headers.customCorrelationKey"
                 release-strategy-expression="size() == 100"/>

1. correlation-strategy just reads some header from message, which you can simply to add on your own
2. release-strategy just checks how how much messages are received by aggregator
3. And in the end <aggregator> returns into output-channel a Collection of your messages' payloads.

That's all

So, you still should know how much messages you need to aggregate. It also can be some Message's header.

[Miko95]

So, I'm force to use a second channel for the Message groups? Is there a way to directly pass the Message group to a service activator?

[Cleric]

No, you can't!
It's anti-pattern too ;-)
MessageGroup is some internal container for specific logic and we don't expose any API to manipulate with it.
It will be a bit confused.

What's problem to manipulate just with List of payloads?

[oleg.zhurakousky]

Can you please explain why do you want to get "more then one Message from the channel at once"? What is your use case?

[Miko95]

@oleg

Since processing one message involves database access, I wanted to improve performance using batch functionality of JdbcTemplate. If I get multiple messages at once from the message channel, I also can update my database for all these messages at once.

@Cleric

I just wanted to know if I can directly pass the List of payloads to my service activator (I used the word MessageGroup because I understood that a MessageGroup was put in the output channel of the aggregator.

[Cleric]

Hello.

if I can directly pass the List of payloads to my service activator

Shortly: yes, you can

[oleg.zhurakousky]

Then its simple
All you need to do is batch up messages in groups and send them to a channel as a single Message - which is a case for aggregation.
I recently presented it in one of the webinars http://www.youtube.com/watch?v=RY6dN...4&feature=plcp
And here is the direct link to the code https://github.com/olegz/s12gx.2011/...ion/aggregator