FTP Connection Support

Hi, I am developing Spring Apps to be integrated with COBOL. I need to use lots of flat file export and import functionalities from Spring Batch which are great.

However, I will need to send the flat files to an FTP server to be processed by COBOL and copy the file to another FTP server for archiving purposes.

Thus, I have wrote a service that can just do that using the Apache Common Net library (org.apache.commons.net.ftp)

Nevertheless, I wonder if there are any developments or existing tools in Spring for this FTP support so that it can integrate seamlessly into Spring.

Some issues that I have encountered
- Testing needs to use external mock library http://mockftpserver.sourceforge.net/
- FlatFileItemWrite requires a write to a resource / file (let me know if it is possible to write to a ByteArray using other ItemWrite class)
Thus, at this moment, I will need to create the file first, send to a couple designated FTP servers then to delete the flat file in the server.
Ideally, I can store the output file in ByteArray (Memory) then write to a couple FTP server just like using Spring Jdbc Data Access
- Working in ZIP ByteArray is preferable as COBOL cannot process delimited flat file easily. Thus, a 6MB fixed length flat file can be easily compressed to 600k to get rid the space padding.

I have solved all of the issues above but I wonder whether there is a better way to handle this using other existing Spring classes

The FlatFileItemWriter has to write to a file if it wants to be safe across failure and restart. It actually doesn't do much else other than manage the file resource - delegating important stuff to do with application logic to FileSetCreator and LineAggregator. You could write your own version using a byte array and the same delegates. The output data would either have to be volatile (not restartable), or else use a file to store across restarts anyway (at which point you might as well just just use the existing features).

There is no Spring equivalent of commons ftp. We actually had an internal discussion about this recently and our Apache guys said that while commons net is not a very active project it is pretty solid - it just works, and there aren't any issues with it, so people just use it.

You can probably avoid having to use the mock ftp server for unit testing just by introducing a strategy layer above the FTP layer (assume that commons ftp works, and just use that as one implementation of a strategy).

There might be some FTP source/target features in Spring Integration soon, but these would be higher level, and would probably use commons net anyway. You can keep track of that in JIRA or via the Spring Integration forum.

Hi Dave,

I agree with you on using the existing FlatFileItemWriter so that it is restartable.

Also, thanks for the tips on unit testing strategy.

My code is used in a production environment for a client in the Netherlands.
So if the FTP features in Spring Integration are implemented, I will implement them in my production environment. I will keep track of that in JIRA.

Finally, I just want to thanks for the good work on Spring Batch. It really helps to speed up the deployment especially since it handles the encoding conversion properly (COBOL is using CP850 for my case).

However, I just had Spring Core training in Singapore 11-14 Mar 08, I just realized that Spring Batch was not mentioned at all. Perhaps, it is a good idea to include a section to introduce briefly on each of other Spring Projects.