Sep 18th, 2012, 08:54 PM
Balancing Load per-client
Sorry if this is obvious or, on the other hand, a complicated topic. Feel free to point me towards books or the like that might cover the topic better.
I'd like to have multiple worker instances running and consuming messages in a round-robin type distribution. Each message should only be processed once. There are a couple of specifics I'm considering as well:
1) If a new feature is added to the system, the old workers will not be able to process it. If I bring up a new worker that has the feature added, the old workers should ignore the message they can't handle and the new worker should get it.
2) A worker instance may be configured to process messages intended for any "client", or configured for specific clients. This would allow us to bring dedicated resources online for a certain "client".
So far in simple tests, I've added a filter in an attempt to satisfy (2), but it seems that messages that are rejected by the filter are lost (which should be expected, since I don't have a discard-channel. Should my strategy be to have the discard-channel feed the messages back up, or is there a better way to filter, before the message is consumed?
I'm sure there are parts of this question that illustrate the areas where I lack understanding.