FTP file download using spring integration

[ftpaccess]

aim is to download a file from ftp unix server to windows machine .
all relevant information is passed on to connect to remote ftp server.
dont see any file getting downloaded
please advice if any mistake .


{code}
port=22
host=ftpldn00
userId=intranet\sysftpgss
password=ftpaccess
LOC_DIR=C:\Ftp_test
REMOTE_DIR=/sysFTPGss/GssTest/Outgoing/DFC/



<bean id="ftpClientFtry" class="org.springframework.integration.ftp.session .DefaultFtpSessionFactory" >
<property name="host" value="${host}"/>
<property name="port" value="${port}"/>
<property name="username" value="${userId}"/>
<property name="password" value="${password}"/>
</bean>



<int-ftp:inbound-channel-adapter id="ftpInbnd"
channel="ftpChannel"
local-directory ="${LOC_DIR}"
remote-directory ="${REMOTE_DIR}"
filename-pattern="*.txt"
auto-create-local-directory="true"
session-factory="ftpClientFtry">
<int: poller fixed-rate="1000"/>
</int-ftp:inbound-channel-adapter>


<int:channel id="ftpChannel">
<int:queue />
</int:channel>

----------------------------------------

AbstractApplicationContext basiccontext = new ClassPathXmlApplicationContext("/reporting-context.xml", ReportStartUp.class);

PollableChannel ftpChannel = context.getBean("ftpChannel", PollableChannel.class);
Message<List<?>> fileList = (Message<List<?>>) ftpChannel.receive();
{code}

[Gary Russell]

Please use [ code ]... [ /code ] tags here (no spaces in brackets), not {code}

I suggest you turn on DEBUG logging and look at the logs. You should see what's happening. There are probably no files that match the criteria, or the files already exist locally.

[ftpaccess]

Thanks for the reply, Gary.


Here is the exception which i am getting. Can you please help? files exist in the remote ftp server at given directory.

2013-03-01 15:58:24,343 INFO [org.springframework.context.support.DefaultLifecyc leProcessor] - Starting beans in phase 2147483647
2013-03-01 15:58:24,345 INFO [org.springframework.integration.endpoint.SourcePol lingChannelAdapter] - started ftpInbnd
2013-03-01 15:58:24,347 INFO [***** IneligibleReportStartUp *****] - ftpChannel ftpChannel
2013-03-01 15:58:25,031 ERROR [org.springframework.integration.handler.LoggingHan dler] - java.lang.IllegalStateException: failed to create FTPClient
at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.getSession(AbstractFtpSessionF actory.java:139)
at org.springframework.integration.file.remote.sessio n.CachingSessionFactory.doGetSession(CachingSessio nFactory.java:100)
at org.springframework.integration.file.remote.sessio n.CachingSessionFactory.getSession(CachingSessionF actory.java:79)
at org.springframework.integration.file.remote.synchr onizer.AbstractInboundFileSynchronizer.synchronize ToLocalDirectory(AbstractInboundFileSynchronizer.j ava:143)
at org.springframework.integration.file.remote.synchr onizer.AbstractInboundFileSynchronizingMessageSour ce.receive(AbstractInboundFileSynchronizingMessage Source.java:144)
at org.springframework.integration.endpoint.SourcePol lingChannelAdapter.doPoll(SourcePollingChannelAdap ter.java:89)
at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :146)
at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :144)
at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller$1.run(AbstractPollingEndpoin t.java:207)
at org.springframework.integration.util.ErrorHandling TaskExecutor$1.run(ErrorHandlingTaskExecutor.java: 52)
at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:48)
at org.springframework.integration.util.ErrorHandling TaskExecutor.execute(ErrorHandlingTaskExecutor.jav a:49)
at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.run(AbstractPollingEndpoint. java:202)
at org.springframework.scheduling.support.DelegatingE rrorHandlingRunnable.run(DelegatingErrorHandlingRu nnable.java:53)
at org.springframework.scheduling.concurrent.Reschedu lingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.cal l(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.jav a:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.access$301(ScheduledThreadPoolE xecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.run(ScheduledThreadPoolExecutor .java:207)
at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl .java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSoc ketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.j ava:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.j ava:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.commons.net.SocketClient.connect(Socket Client.java:176)
at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.createClient(AbstractFtpSessio nFactory.java:152)
at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.getSession(AbstractFtpSessionF actory.java:136)
... 22 more

[Gary Russell]

Your configuration is incorrect...

Caused by: java.net.ConnectException: Connection refused

This means ${host} is not listening on ${port}

[ftpaccess]

Thanks a lot Gary. that helped alot and saved time.

Can you please suggest a approach for following scenario?
I get files from remote directory using FTP.

Remote directory :
tblCCLCounterparty_20130122_1536.txt
tblCCLCounterparty_20130202_1530.txt
tblCCLCounterparty_20130203_1236.txt

how can i select latest file or say todays file ? Is there a way i can add a selector or how to build one ?

my config so far is as follows :

<bean id="ftpClientFtry" class="org.springframework.integration.ftp.session .DefaultFtpSessionFactory" >
<property name="host" value="${host}"/>
<property name="port" value="${port}"/>
<property name="username" value="${userId}"/>
<property name="password" value="${password}"/>
</bean>


<int-ftp:inbound-channel-adapter id="ftpInbnd"
channel="ftpChannel"
local-directory ="file:${LOC_DIR}"
remote-directory ="${REMOTE_DIR}"
filename-pattern="*.txt"
auto-create-local-directory="true"
session-factory="ftpClientFtry">
<intoller fixed-rate="1000"/>
</int-ftp:inbound-channel-adapter>

please advice

[Gary Russell]

When posting code/config, please use [ code ] ... [ /code ] tags (no spaces in brackets).

Yes, implement a FileListFilter (for type: FTPFile). You can then choose which file(s) to retrieve.

Declare your filter as a <bean/> and provide a reference to it with the filter attribute on the adapter. You can't use a pattern as well as the filter so you'd have to add to code to select only *.txt files as well. You can do that in-line, or use a CompositeFileListFilter to wrap your filter and a FtpSimplePatternFileListFilter.

[ftpaccess]

thanks for all the help so far.

Can you please help me with the following ?

1. failed to send message to channel 'ftpChannel' within timeout: -1
I see this message when file is downloaded using ftp . { ftp:inbound-channel-adapter } . the file gets downloaded , but i see above error. i am not sure what its trying to tell me and how do i avoid it ?

2. how can i configure org.springframework.batch.item.file.FlatFileItemRe ader to look into the directory where ftp files are downloaded and start reading them for further processing.
i have set resouce property to following

 

reading process starts with this . 

<bean id="productReportReader" class="org.springframework.batch.item.file.FlatFileItemReader">
		<property name="resource" value="file:C:\dev\workspace\eclipse 360\Report\Ftp_test\" />
		<property name="linesToSkip" value="1" />
		<property name="recordSeparatorPolicy" ref="productRecordSeparatorPolicy" />
		<property name="lineMapper" ref="productLineMapper" />
		<property name="strict" value="false" />
	</bean>


ftp process

<bean id="ftpClientFtry"  class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory" >
		<property name="host" value="${host}"/>
		<property name="port" value="${port}"/>
		<property name="username" value="${userId}"/>
		<property name="password" value="${password}"/>
	</bean>

<int-ftp:inbound-channel-adapter id="ftpInbnd"  
		channel="ftpChannel"
		local-directory ="${LOC_DIR}"
		remote-directory ="${REMOTE_DIR}"
		filename-pattern="*.txt"
		auto-create-local-directory="true"
		session-factory="ftpClientFtry">
		<int:poller fixed-rate="1000"/>
	</int-ftp:inbound-channel-adapter>

[Gary Russell]

You need to have something subscribed to channel 'ftpInbnd'.

For launching Spring Batch jobs, take a look at the JobLaunchingMessageHandler in the Spring Batch Admin project.

[ftpaccess]

with file list filter , i can select files with extention / dates etc . thanks for the suggestion.

[ftpaccess]

Regarding , You need to have something subscribed to channel 'ftpInbnd'.
created a service-activator as follows. dummy class messagehandler has a few public methods.

<int-ftp:inbound-channel-adapter id="ftpInbnd"  
		channel="ftpChannel"
		local-directory ="${LOC_DIR}"
		remote-directory ="${REMOTE_DIR}"
		filter="fileFilter"
		auto-create-local-directory="true"
		session-factory="ftpClientFtry">
		  <int:poller fixed-rate="1000"/>  
	</int-ftp:inbound-channel-adapter>

	<int:channel id="ftpChannel">
		<int:queue capacity="1000"/>
	</int:channel>
	
	<int:channel id="outChannel" >
		<int:queue capacity="1000"/>
	</int:channel>

	<int:service-activator  input-channel="ftpChannel" output-channel="outChannel" ref="msgHandler" />
	<bean id="msgHandler" class="com.util.MessageHandler"/>

this gives error saying
org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: No poller has been defined for endpoint 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0', and no default poller is available within the context.

Caused by: java.lang.IllegalArgumentException: No poller has been defined for endpoint 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0', and no default poller is available within the context.

my aim to create service-activator was to get rid of
org.springframework.integration.MessageDeliveryExc eption: failed to send message to channel 'ftpChannel' within timeout: -1

please advice