Parallel downloading of files from FTP servers

[mdond]

Reading your comment though I'm wondering if maybe just synchronising receive() would be good enough for now?

This would synchronize the downloading of the many small files, so it would behave like downloading with a single thread. This way the concurrent scenario of the poller to support many small files wouldn't optimize network usage, it only would require more threads without any benefit. Is this concurrent poller scenario really useful? Probably in most cases different ftpSources have to be handled, these should be polled concurrently, but I don't see a real case for the same FTPSource polled concurrently. I am not sure, if this was the intention by ramkris.

[iwein]

Well the concurrent poller scenario goes for all sources, it has its uses in other areas. I fully agree that if I synchronize receive there will be no benefit in concurrent polling on the source side. I interpreted ramkris' post as a request for downloading in multiple threads in the same source.

@ramkris, are you still following this? (now would be a good time to nudge things your way )

[mdond]

However it wouldn't be enough to synchronise receive(). Its the combination of receive() and onSend() that must be synchronised.

[iwein]

You're absolutely right, we need a monitor of some sort.