Oct 26th, 2009, 03:18 PM
file:inbound adapter - functioning in clustered environment
How does the file inbound channel adapter function in clustered environment? can multiple instances pick the same file.
If we use LockFileFileListFilter, will multiple instances try to get a lock on the same file.
We observed that the filterFiles in AbstractFileListFilter , prepares a list of accepted files before trying to lock individual file, so in such scenario how would it be handled if multiple clusters are running in parallel and get the same file list.
Oct 26th, 2009, 03:36 PM
I haven't tested this in great detail, but with the nio based locking and some luck with the underlying file system you should be able to lock files across multiple jvm's. The lock file based mechanism has a (small) window of vulnerability where a file can be picked up multiple times (if mv is not atomic).
I once jokingly called file locking unholy, and the reason for this is that it gets complex real quick in multy vm concurrent scenarios. If you have the time and resources to use it in anger I'd be *really* happy with any test results you can provide. Obviously I will prioritize any bugs you might find in the process.
Oct 26th, 2009, 05:11 PM
I have launched two standalone programs monitoring the same input directory for files and each one gets a lock on it before parsing and doing further processing.
So for test as both were running, i manually copied some files on to the directory. There was some overlap on the list of files both tried to process but only one process locked it. Which appears to be working in this case but looking at the timestamps both processes tried the lock at different times. so the first one won i guess. Due to limitations in my environment setup (multi processor almost processing at same milli second) i couldnt make them try to lock at same time.
Hope to see it sometime.
Feb 11th, 2010, 05:12 PM
I too have a need for this because we are deploying applications in a clustered environment with file polling capabilities on the same mount directory.
I noticed the LockFileFileListFilter class was delete from the trunk. What are we suppose to use to accomplish the file locking capability? Are there any documentations on this?
I saw a JIRA issue (http://jira.springframework.org/browse/INT-821) about adding namespace support for file locking, but the usage is still not very clear.
Last edited by jin1212; Feb 11th, 2010 at 05:20 PM.
Feb 12th, 2010, 03:37 AM
I haven't documented the usage for 2.0.0.M2 and it's probably going to change for 2.0.0.M3. Also the implementation is going to change in a refactoring that I've planned in INT-947. I've created http://jira.springframework.org/browse/INT-984 and added some links so you can understand the dependencies better.
At this moment your best bet is to look at the test cases for INT-821 which you can find here:
I hope that helps.
Feb 12th, 2010, 07:53 AM
Thank you! I will take a look at the test cases
Feb 12th, 2010, 08:39 AM
Please remember that this code is still in flux. I'm quite sure the namespace is going to change after INT-947, possibly breaking the M2 support. Of course it's just great that you're trying it out
Mar 2nd, 2010, 11:35 AM
Just checking about cluster support for inbound channels? is it going to be part of a future release
Jan 3rd, 2011, 04:07 AM
I'm also interested in this functionality. Any plans for future release?
Originally Posted by srinivas_vsk
Jan 3rd, 2011, 05:15 AM
Just remind me: what exactly is missing for cluster support? Can you provide some examples (preferably in the form of running code)?