Is message bus the point of failure ?

Hi,

Can you tell me if the Message Bus of Spring - Integration framework is :
- a in memory messages carrier between endpoints ?
- a point of failure for message delivery ?

Ex :
1) Message channel = directory where files of orders are received
2) Parsing file endpoint A = POJO class used to read to files received and generate for each order an order message send to the message channel through the bus
3) Order verification endpoint B = POJO class verifying the order validity from messages received through the Message channel - bus

If during the transfer of the messages from endpoint A to endpointB, the application running the spring framework crashes, does it mean that the in memory messages carried out by the Message channel - Bus are lost ?

To avoid this situation, would we have to use a JMS server to transfer the messages from endpointA to endpointB in order to guaranty QoS and recover of the message in case of crash ?

Regards,

Charles

Charles,

The current (milestone 1) version of Spring Integration does not offer reliability for internal MessageChannels, so yes messages can be lost between endpoints. Currently a JMS source adapter can be configured with any of the acknowledgment options (including 'transacted' Session) so that an inbound JMS Message will not be removed from the queue/topic unless the message is successfully sent to a Spring Integration MessageChannel. Furthermore, you can create a reply MessageChannel to wait on, set it as the "returnAddress" of a Message that you are sending, then acknowledge the received JMS Message only after the Spring Integration reply Message is successfully received (which may span internal MessageChannels and endpoint invocations). Those are essentially the limits of reliability at the moment, but we are working on a number of things in order to provide support for the full spectrum of reliability vs. performance. The milestone 2 release (this week) will likely provide basic transaction-demarcation at the MessageEndpoint, and then much more work is scheduled for the m2-m3 timeframe (roughly the next month).

Here are a few of the ideas we are considering and/or prototyping. Feedback would be greatly appreciated!

1) a ChannelInterceptor that delegates to a pluggable MessageStore strategy. The channel.send() method will not return until the message has been successfully stored, and the message will then be removed from the MessageStore prior to returning the message from the receive() call. Note that this will impede performance and throughput, but if using JDBC, then the transaction can span a JMS receive and JDBC insert... if the insert fails, the transaction will rollback, and the JMS Message will not be removed from the queue/topic.

2) ability to invoke an endpoint directly within the caller's thread - and thus in the same transaction if active.

3) grouping transactions so that a transaction bound to the thread of the original sender will block on commit until the transaction(s) of the downstream receiving thread(s) are also ready to commit.

4) support for a JMS-based MessageChannel implementation as an alternative to the current default of SimpleChannel.

-Mark

Mark,

Many thanks for your reply. Concerning your ideas, here is my point of view :

1) a ChannelInterceptor that delegates to a pluggable MessageStore strategy.

>> This solution seems to me the most appropriate even if the worload to develop such feature or to implement it will be perhaps higher than the three others. Nevertheless, this architectural design offers the possibility for the developer point of view to be independent of the underlying bus in charge to deliver the messages from one endpoint to another. Another advantage that I see, is that Spring intergration could propose or support different existing solutions (ex : Normalized Message Router) or specifications like JBI, SCA and propose to the developer to choose the solution ( = Message Store) that he/she prefers through IoC.

2) ability to invoke an endpoint directly within the caller's thread - and thus in the same transaction if active.

>> Is it a good idea to allow to declare a caller's thread as transaction active ? This is probably interesting when the developer can isolate in the code the tasks where transaction/commit must be handled but when the number of tasks/filters (according to enterprise pattern defintion) will increase, How will it be able to manage all the transactions even specifically if a transaction must be propagated through several filters ?
From my point of view, the transaction must be defined from a Service point of view and perhaps more precisely at the level of the POJO classes but not for the messages. If the messages must persist between different endpoints, this is a infrastructure responsability and must be defined in the JMS channel.

3) grouping transactions so that a transaction bound to the thread of the original sender will block on commit until the transaction(s) of the downstream receiving thread(s) are also ready to commit.

>> See previous answer.

4) support for a JMS-based MessageChannel implementation as an alternative to the current default of SimpleChannel.

>> This is a good idea and the phylosophy of this approaches meet the point 1). Depending of the development to be done, the developer must have the possibility to choose from a in-memory bus (= Simple channel) to a JMS bus where he/she can acrivate or desactivate the persitence in order to guaranty the reliability of the platform.

Regards,

Charles

Has any of this proposed functionality made it into a release? I checked the m3 changelog and did not see any of these enhancements.

i'm so interested on adding reliability to channels, but i don't understand the approach at all, so it would be nice to see a little example of the use of MessageStore (i can't see anything in reference doc.)

Thanks in advance.