publishsubscribechannel when exception gets thrown in one of the subscribers

[kbohnenberger]

What is the expected behavior of the publishsubscribechannel when a runtime exception gets thrown by one of the subscribers? Should all of the other subscribers still get the message?

For instance, if there are 3 subscribers to a publishsubscribechannel and one of the subscribers throws a runtime exception, is the expected behavior that the other two subscribers will still receive the message and process normally or is the expected behavior that none of the subscribers will receive the message?

The latter appears to be the case for me but I need to do some more testing to help me verify what is going on.

Thanks

Keith

[Mark Fisher]

Currently, it does fail fast as soon as an Exception occurs. That does not affect other handlers if the pubish-subscribe-channel has a TaskExecutor reference. But, in the default case where there is no TaskExecutor, that will prevent the remaining handlers from being invoked.

I have thought about this recently, since the RecipientListRouter does have an "ignore" flag for failures. Perhaps we should provide the same thing for publish-subscribe-channel. It would be FALSE by default so that the current fail-fast behavior is the same, but at least that would enable switching it off. Does that make sense?

[kbohnenberger]

Adding the "ignore" flag for failures sounds good.

With that said ....
I think I'm missing something.
I'm a little confused about what the purpose of adding a task-executor to the publishsubscribechannel would be?

A message comes into the publishsubscribechannel, the message gets "sent" to each subscriber of the publishsubscribechannel. What is a use case for adding a task-executor to this functionality?

[Mark Fisher]

When a Message is sent to a publish-subscribe-channel, it uses a dispatcher to invoke the MessageHandlers that are subscribed to that channel (e.g. any service-activator, transformer, etc.). By default, that invocation will happen in the same Thread. By adding a TaskExecutor you get the advantage of letting those dispatches occur concurrently, but the tradeoff is that transactional context is not shared between the sender and the handler(s).

Is that a clear enough description?

[kbohnenberger]

So if I would add a org.springframework.scheduling.concurrent.ThreadPo olTaskExecutor I can make the publishsubscribechannel multi-threaded ... at the risk of complicating the "transaction?" Is that right? If so, would the "incoming transaction" essentially end at the publishsubscribechannel?

[Mark Fisher]

Yes, that is the idea. In such a case, the only thing that can affect the sender's transaction is if a TaskExecutor throws an Exception to reject the task itself.

[Mark Fisher]

I created this issue:
http://jira.springframework.org/browse/INT-734

I just resolved it also. The 'ignore-failures' attribute will be available in Spring Integration 1.0.3.

[kbohnenberger]

Mark

Do you have any suggestions for how to handle the durability issue related to the publisher-subscribe-channel?

For example, my flow gets a message in, then goes through a number of common steps which end with the messages "published" to a publish-subscribe-channel. If at this point my "incoming transation" is over right? So if my incoming message was pulled off of an "incoming Queue" the acknowledge will get sent back to the JMS server and the message will come off of the queue. If my stand alone java process gets shut down before all the "listeners" of the publish-subscribe-channel" are done, I will lose messages.

Any suggestions on how I handle this durability problem?

Thanks

Keith

[Mark Fisher]

Are the listeners on the publish-subscribe-channel executed in the same thread that receives the JMS message? i.e. are you using the "task-executor" attribute on the publish-subscribe-channel?

[kbohnenberger]

At the moment I am not using the task-executor but I am weighing the benefits/risks of doing that which is what drove my question.

It seems that I have 3 options:
1) Use publish-subsribe-channel with ignore-failures = false
In this case the issue is that if one of the "subscribers" throws an exception, the transaction will fail and the incoming message will stay on the queue. So if the 3rd of 5 subscribers throws the exception, the first two subscribers got the message but the 3,4,5th subscribers did not process the message.

2) I can use publish-subscribe-channel with ignore-failures=true
In this case all my subscribers will process the message and only the subscriber who throws the exception will not process the message. The issue here is that I have to come up with a way to deal with the "problem subscriber" and figure out how to "save" the message in order to fix the problem and retry it later. Since ignore-failures was true the transaction will complete even if there is an exception in one of the subscribers and the incoming message will come off of the incoming queue.

3) Add the "task-executor" capability
In this case the benefit seems to be that a "slow" subscriber will not hold things up for the other subscribers. The problem area is that the I believe the transaction will complete once the message gets put on the in-memory executor queue. Once the transaction completes, the message will come off of the incoming queue. The danger in this seems to be that if my stand alone spring integration process gets shut down for any reason I will lose the messages that were on the in memory executor queue.

Do I have all of that right? If so, I'm trying to way which one is the best case for my needs.