JPA Poller with concurrent threads

[Mohit Madan]

We have a use case where we need to read the records from the table based on a flag and process these records basically sending to various web services. We need to select and update the flag in the single transaction also support concurrent threads so that the poller only picks the record only once

We were planning to use the JPA adapter for polling, as per the JPA inbound adapter document there is no support for select and update in a single transaction as the way it is supported by JDBC inbound Adapter. Also we need to scale this by using task executor supporting multiple concurrent threads.

Does this mean JPA poller doesn't support select and update in single transaction as supported by JDBC adapter and the only way to do this is to use JDBC adapter only

[Cleric]

Hi!

<jpa:inbound-channel-adapter> supports property delete-after-poll="true". The entityManager.remove(entity) causes at the same transaction.
However if you don't interested in real 'DELETE FROM...' you can override this operation with custom UPDATE:
http://stackoverflow.com/questions/9...-jpa-hibernate

Take care,
Artem

[Mohit Madan]

Thanks Artem, this is helpful

[Mohit Madan]

Hi Artem,

Just a follow up question, in case where we would require horizontal scaling for the JPA poller would it be safe that the JPA poller picks the record only once. Do we need to any specific configuration for this?

[Cleric]

Hi!

Do you mean something like this:

<int-jpa:inbound-channel-adapter entity-manager="entityManager"
						 jpa-query="from Foo"
						 max-number-of-results="10"
						 delete-after-poll="true"
						 channel="processFooChannel">
	<int:poller fixed-rate="1000" task-executor="threadPoolTaskExecutor" max-messages-per-poll="10">
		<int:transactional/>
	</int:poller>
</int-jpa:inbound-channel-adapter>

?
Than you really need this one:

@SQLDelete(sql = "update Foo f set f.deleted = true where f.id = ?", check = ResultCheckStyle.COUNT)

In this case, if some another transaction tries to get the same Entity, it ends up with StaleObjectStateException.

HTH

[Mohit Madan]

Hi,

So would this transaction also be managed across 2 different nodes of server. So for example I need to do horizontal scaling so there would be 2 nodes of server with multiple threads running on both the server which would have their individual poller which are polling to the same database table. So would JPA poller manage this in transaction in the same way

[Cleric]

Hi!

Sorry for late reply.

So, my opinion:

1. Make a transaction of the <jpa:inbound-channel-adapter> as short as possible.
2. adapter should read one entity per poll - "maxNumberOfResults = 1"
3. "channel" should be some thread-separated one (queue, executor-channel etc.)

In this case, if your transaction ends up with StaleObjectStateException, it won't affect to other entities in other transactions.
It would be better to use SELECT ... FOR UPDATE SKIP LOCKED, but it is only for Oracle, and, as you see, there won't benefit to use JPA-adapter.
Of course, you can use for this solution "native-query" or "named-query" attributes.

Take care,
Artem