Nov 16th, 2011, 02:36 PM
Polling multiple directories in parallel
Please provide your ideas for a robust and clean design for the following need.
There are 5 different folders, each representing an inbound data type (.txt files) in tabbed delimited format. I want to start the program that would poll the folders and if a file is available, it would be processed. Would like to process the files in parallel.
1. Spring Integration - to poll for availability of files
2. If files are available, invoke a Spring Batch job
3. The batch job would read the file, process it and write it to suitable target
All the three steps to happen in parallel for the different data types.
1. As the Service Activator would be mainly responsible for getting the File objects and launching a Spring Batch job, do I need to create multiple Service Activator configurations, one for each data type or is there any scope of reusing a single Service Activator configuration?
2. Since there is a need to process feeds in parallel, would a AsynchronousGateway help here?
3. As there are 5 different directories to be polled, I would need to create 5 different inbound-channel-adapter configurations to hold details of each of the folders and the polling frequency. Please correct me if I am wrong.
Appreciate your thoughts.
Tags for this Thread