Feb 25th, 2011, 07:59 PM
Newbie: Architecture Question using AMQP
I've been doing lots of reading about AMQP and messaging and am a little confused on how to use such a technology. Here's the scenario I'm looking at:
1. Some client (web, mobile, etc.) wants to send some text into the database. The system needs to encrypt the text before inserting it into the database.
2. Because encryption is slow, I want the system to asynchronously send the data to a separate encryptor so the client doesn't sit there and wait and so the system doesn't slow down to a crawl.
So my logical choice is to use AMQP and RabbitMQ. So, my main question is how are the consumers typically written? Are they written within web apps and run on a servlet container or are they typically just threads that you run from a standalone that just sit there and wait for some message to come along on the queue? Would you run multiple different consumers on the same servlet container?
client --> web app/producer (to receive the message and send to the queue) --> RabbitMQ --> consumer (what's the most efficient and best way to write this? Web app or just some standalone jar?)
Thanks for your help!
Feb 26th, 2011, 07:15 AM
I would suggest to also look at Spring Integration project and AMQP adapters. This way the request can go to a custom transformer configured as asynchronous endpoint allowing it to return right away while request continues to process.
Feb 28th, 2011, 01:09 AM
I understand that part, but I'm curious what the most efficient and best way is to actually run the endpoints. Do you run them in a web container or are they simply standalone classes with a main method? So say I had 15 endpoints all doing different things, is it best to have them all as standalones or are they better to be within a single WAR file being served by a web container?
Feb 28th, 2011, 06:59 AM
That's a good question. Unfortunately without knowing more details about your app it is hard to answer. The upside of them being in a single WAR is that they are all running within the same application context. However the downside of it is that by shutting down the WAR you shutting down all of them. So the question you must ask is do any of your endpoints have a life of its own or are they all a part of one app. If so, then single WAR would be OK. And of not then you might also have to also worry about cross-application-context communication and that is where Rabbit would be even more helpful.
In other words in the context of your app and how it is distributed you can look at SI as an optimization technique where you can wire a pretty complex processes within a single JVM and use AMQP inbound/outbound adapters for cross-platform and cross-jvm collaboration.
Feb 28th, 2011, 05:51 PM
Cool, that makes sense, Oleg.
I guess the next logical question is performance-wise how 15 separate endpoints scale vs. 15 endpoints running in a web container. That will have to be something I test out eventually I guess. My only thought here is that the benefit of standalones is that when they're done, they're done and the memory is released. With the web container, it's possible for the cleanup to not be as pretty or some memory leak in the container. Just a quick thought....