Multi channel endpoint and extending AbstractPollingEndpoint

[Mark Fisher]

I was just trying to explore other options that might be simpler in terms of the configuration. At some point (once we have a GA version available, planned for Spring Integration 2.1), you might want to consider our AMQP integration.

In the meantime, the way you are describing it might handle it fine. Another option I would consider is simply referencing a "task-executor" within the <poller> element. Then, you could have a number of different, reusable thread pool configurations. For example, you might have "high", "medium", and "low" priority (where priority is handled by the number of threads). Then you would be able to use "highPriorityExecutor" everywhere you need that one.

-Mark

[keidi19]

Hi Oleg,

I'm not sure I understand your post. I don't have any trouble making the queue distributed with HazelCast. The ref to "DistributedQueue" in the configuration excerpt above is already pointing to a HazelCast distributed queue.

Shahar

[oleg.zhurakousky]

Good, i wasn't sure you were aware of that, my bad

So, now since your queue is distributed I am still not sure as two why it could not be prioritized. I understand that you have a high throughput system and that you may have so many high-priority messages that low-priority messages will remain in the queue indefinitely. But IMHO that is exactly what has to happen in the world where we have high/low priority.
However I am NOT advocating that your low-priority messages should remain there forever. There are two ways I see how you can handle this today.

1) - by tuning up your poller and task executor configuration according to the load in your system - basically making your poller poll for messages just a bit faster then the *average* frequency of messages coming in.

2) - PriprityChannel can be configured with custom Comparator where you can implement your own compare logic that might take timestamp of the message into consideration and UP the priority of the message if it has been sitting for a certain amount of time. So the low-priority message after sitting in the queue for a certain period of time can essentially be upgraded to a higher priority by simply re-ordering such queue.

what do you think?

[keidi19]

Mark,

Thanks for the task-executor idea (actually it reminded me that I forgot to assign my task-executor ).

Regards,
Shahar

[keidi19]

Hi Oleg,

The thing is that in our system nothing is really low-priority but rather "slightly-lower" .
I really need a more fine tuned adjustment of the way items are prioritized. Using variable rates, variable intervals and different task-executors might do the trick.

I'm a bit concerned about using a sophisticated comparator. I agree that taking into account the message's timestamp might do the trick but I'm not sure if this is good enough. For now I'll stick to the current solution and see if it works for us.

Thanks for all the ideas. It really helped me.

Shahar