Messaging question

[tentacle]

This is a rhetoric question, so you don't need to bother replying. It's just that I've been thinking a lot about how to integrate messaging into Spring and I haven't found the answer yet. The big problem is pairing database and JMS transactions to get guaranteed delivery/rollback. The obvious solution is global transactions, which I've used successfully in the past. For a variety of reasons this is no longer an option for me, the best reason being that I cannot afford global transactions anymore.

Spring provides a very elegant replacement for stateless session beans, and claims it cannot replace other EJB artifacts, including MDB's, which is probably correct.

In fact, combining two transactions, regardless of their types, is delegated by Spring to JTA, for all the right reasons. However, at least to me this does not mean Spring is not fit to provide an answer for my messaging question. Actually, the answer is as perfect as it is simple: rethink the messaging requirements, shift the messaging paradigm and do everything in one transaction: the database transaction.

Shifting the paradigm could mean: shifting from guaranteed delivery to best effort delivery. Best effort delivery does not necessarily mean try once and if that fails don't try again. It could mean: create the message and keep retrying until delivery, which may never occur.

Creating the message could mean storing the message in a database table first, then start the first delivery attempt. Delivery then means confirmed delivery. This would require the client to be first of all idempotent, secondly to actually confirm every message it receives to the sender. Until then the sender should continue resending, keeping in mind a realistic timeout. Once the message has been received a callback could be invoked, to continue or complete the business transaction.

If I'm not mistaking, whether the message has been successfully send and delivered or not is no longer important when working inside a database transaction, it's the registration of the message that's important.

I was just wondering if other people out-there have come up with any solution that addresses the delivery problem differently than global transactions.

[katentim]

Actually, the answer is as perfect as it is simple: rethink the messaging requirements, shift the messaging paradigm and do everything in one transaction: the database transaction

I've thought the same thing. You just have to come up with a relatively portable way storing messages on different DBs - shouldn't be hard. You can then easily add rules to define re-tries, and alerts on delivery failure.

[tentacle]

I've thought the same thing. You just have to come up with a relatively portable way storing messages on different DBs - shouldn't be hard. You can then easily add rules to define re-tries, and alerts on delivery failure.

Storing the messages indeed has to happen in pair with the transaction manager configured for the business method in question, it has to happen under his wings.

I'm not sure delivery failure is an important issue. If you're talking about messaging via e-mail, P2P networks or via SMS, any delivery failure should be reported by the transport itself to the administrators, not necessarily through the messaging infrastructure. In my opinion the outcome of sending a message is unresolved until an answer is received or the pending message is cancelled. I presume confirmation of delivery is important for every type of business process, even if a response is not required for the actual process.

How would you define rules for retries?

[katentim]

How would you define rules for retries?

Number of tries before failing. Time between re-tries - maybe increasing with each attempt.

[tentacle]

How would you define rules for retries?

Number of tries before failing. Time between re-tries - maybe increasing with each attempt.

I've been looking at Mule recently and it integrates nicely into Spring. You can publish a MuleApplicationEvent on the application context that contains the message and a endpoint, which it catched by Mule for delivery.

A similar event type could be used that can be published by any business component and catched by a listener that stores the message plus meta-data in the DB, within the current DB transaction. After the transaction completes, a seperate thread, probably launched by Quartz, can fetch the messages that are due for (re)sending, wrap the messages in a POJO per message implementing a specific interface, creates MuleApplicationEvents for them and publishes them. The wrapping happens transparantly.

The receiver receives the POJO as a message, probably via Mule. An interceptor (probably extending DelegatingIntroductionInterceptor) wraps the business component and provides a single-parameter method that accepts the interface type of the POJO wrapper. The method writes the unique id of the message + the unique id of the sender to a DB table, unwraps the actual message and passes this message to a method of the business component that has a single parameter of the type of the message. The method of the interceptor should be demarcated declaratively by a suitable transaction manager. The POJO is then propagated through Mule which attempts to return it to the sender. This should only attempted once per incoming message. If a message arrives more than once, the interceptor will notice and just propagates the message through Mule.

If the business method of the receiver throws an exception, the interceptor should determine if this is a business exception (through declaration) and if so return the exception to the sender. If the business methods returns a value, this too should be returned to the sender. Both exception and return value should be kept in the DB and should be set on any idem incoming message before returning to the sender.

But what about a callback for the sender? How will exceptions thrown by the receiver be handled?

I'm gonna get a drink now.

[Ben Alex]

If using Mule you've probably already stumbled across it, but the book, Enterprise Integration Patterns, is a very thorough discussion of messaging patterns. There is also a web site where you can freely read about many of the patterns: http://www.enterpriseintegrationpatterns.com.

[croco]

Hi,

As I understood you need some sort of transaction between some database operations and JMS operations. You don't necessarily need distributed transactions to do this. If the JMS messages get persisted in the same database, most application servers optimize the 2pc transaction into a simple transaction over the same underlying JDBC connection.
This depends on your JMS implementation; but if you could make your JMS use the same JDBC datasource as your JDBC/Hibernate transaction manager, you would have the same results.

Regards, Croco