file:inbound adapter - functioning in clustered environment

[srinivas_vsk]

How does the file inbound channel adapter function in clustered environment? can multiple instances pick the same file.
If we use LockFileFileListFilter, will multiple instances try to get a lock on the same file.
We observed that the filterFiles in AbstractFileListFilter , prepares a list of accepted files before trying to lock individual file, so in such scenario how would it be handled if multiple clusters are running in parallel and get the same file list.

Any thoughts?

Thanks
Srinivas

[iwein]

I haven't tested this in great detail, but with the nio based locking and some luck with the underlying file system you should be able to lock files across multiple jvm's. The lock file based mechanism has a (small) window of vulnerability where a file can be picked up multiple times (if mv is not atomic).

I once jokingly called file locking unholy, and the reason for this is that it gets complex real quick in multy vm concurrent scenarios. If you have the time and resources to use it in anger I'd be *really* happy with any test results you can provide. Obviously I will prioritize any bugs you might find in the process.

[srinivas_vsk]

I have launched two standalone programs monitoring the same input directory for files and each one gets a lock on it before parsing and doing further processing.
So for test as both were running, i manually copied some files on to the directory. There was some overlap on the list of files both tried to process but only one process locked it. Which appears to be working in this case but looking at the timestamps both processes tried the lock at different times. so the first one won i guess. Due to limitations in my environment setup (multi processor almost processing at same milli second) i couldnt make them try to lock at same time.

Hope to see it sometime.

Thanks

[jin1212]

I too have a need for this because we are deploying applications in a clustered environment with file polling capabilities on the same mount directory.

I noticed the LockFileFileListFilter class was delete from the trunk. What are we suppose to use to accomplish the file locking capability? Are there any documentations on this?
I saw a JIRA issue (http://jira.springframework.org/browse/INT-821) about adding namespace support for file locking, but the usage is still not very clear.

[iwein]

I haven't documented the usage for 2.0.0.M2 and it's probably going to change for 2.0.0.M3. Also the implementation is going to change in a refactoring that I've planned in INT-947. I've created http://jira.springframework.org/browse/INT-984 and added some links so you can understand the dependencies better.

At this moment your best bet is to look at the test cases for INT-821 which you can find here:
http://jira.springframework.org/brow...eye-issuepanel

I hope that helps.

[jin1212]

Thank you! I will take a look at the test cases

[iwein]

Please remember that this code is still in flux. I'm quite sure the namespace is going to change after INT-947, possibly breaking the M2 support. Of course it's just great that you're trying it out

[srinivas_vsk]

Just checking about cluster support for inbound channels? is it going to be part of a future release

[krzychu]

Just checking about cluster support for inbound channels? is it going to be part of a future release

I'm also interested in this functionality. Any plans for future release?

[iwein]

Just remind me: what exactly is missing for cluster support? Can you provide some examples (preferably in the form of running code)?