JPA inbound-channel-adapter and max-messages-per-poll

[krzychu]

Hi SI users,

I have configured JPA adapter as follows:

<int-jpa:inbound-channel-adapter id="pendingJobsJpaAdapter" channel="inputChannel"
		entity-manager-factory="entityManagerFactory" named-query="fetchJobsReadyForProcessing" max-number-of-results="10">
		<int:poller fixed-rate="5000" max-messages-per-poll="1" />
	</int-jpa:inbound-channel-adapter>

and I expected tyat 1 JPA entity will be send to inputChannel as payload and I was wrong. The inputChannel receives List of 10 entities (all returned by query) as a payload. Do you know how I can limit the payload to one message?

The second question is how to configure task executor with pool-size="10" so each thread will be processing one entity (10 will be processed in parallel).

Thanks,
Krzysztof

[Cleric]

Hi!

Let's solve your issues one by one:

Do you know how I can limit the payload to one message?

just max-number-of-results="1"

to configure task executor with pool-size="10"

<task:executor id="threadPoolTaskExecutor" pool-size="10"/>

10 will be processed in parallel

<splitter outbound-channel="entityProcessChannel"/>

<channel id="entityProcessChannel">
	<dispatcher task-executor="threadPoolTaskExecutor"/>
</channel>

Now a question: does it make sense to limit jpa:inbound-adapter to fetch 1 entity, if you can simply process each via splitter and executor channel?

Take care,
Artem

[krzychu]

<splitter outbound-channel="entityProcessChannel"/>

<channel id="entityProcessChannel">
	<dispatcher task-executor="threadPoolTaskExecutor"/>
</channel>

Now a question: does it make sense to limit jpa:inbound-adapter to fetch 1 entity, if you can simply process each via splitter and executor channel?

Take care,
Artem

Thanks for this fast response. It looks like a splitter is what I was looking for As for the jpa:inbound-adapter I did not want to set the fetch limit to 1, I was refering to max-messages-per-poll defined on poll component. Do you think that max-messages-per-poll on pooler makes any sens when limit on JPA max-number-of-results is set?

One more question. Is there any difference between setting task-executor on the poller level and setting it on the channel level (as you did <int:dispatcher ....)?

Thanks,
Krzysztof

[Gary Russell]

MMPP is usually only relevant for MessageSources that emit messages one-at-a-time. As you have found, the JPA adapter creates a list that can be busted apart into single messages with a simple <splitter/>.

If, say, there are 100 objects but you limit the max results to 10 and you set MMPP to 10 (or higher); you'll get 10 messages with 10 each. However, this could cost 10 round trips to the database. You'd be better off just getting all 100 in a single call.

Hope that's clear.

As far as the task executor is concerned - it's all about when the message is handed off to another thread.

In Artem's example, the handoff is done AFTER the splitter - this means that each individual message will processed in parallel (up to the pool size).

By putting it on the <poller/>, you'd do the handoff before the spliiter, so you wouldn't get any parallel processing - each "split" message would be handled on the same thread (serially).

Of course, if you want all the processing to occur in a single transaction, then you must not handoff to another thread.

[krzychu]

Thanks Gary for this great explanation. I do want to process my messages in parallel so I will use splitter->channel-with-task-executor->(the rest of the SI flow).