After some time the poller stops polling using ftp spring integration

[Kruti Shah]

My Requirement
1.Need to pick up files from three different folders in the same directory on the remote ftp as and when generated(Eg.a/b/new,a/c/new,a/d/new) and put it in the corresponding folders on local machine and delete the files from remote after copying to local.
2.Do some processing on local machine and move the files in a different folder so as to avoid synchronization problem.
3.Put the files back to remote ftp on(Eg. a/b/archive,a/c/archive,a/d/archive)
*So i have used three pollers to do so
4.Also I need to put a file from local as and when generated to remote ftp in some folder.-so I have used the 4th poller.

1.Exception which comes
in ErrorHandler
Localized Message:Problem occurred while synchronizing remote to local directory
Cause:Connection refused: connect

*in debug mode in the Thread dump.
some threads stop at method-unsafe.park(boolean,long)
some threads stop at socketInputStreamread() something like that

I have attached my context file.Please advise.

Thank You!

[Kruti Shah]

I m sorry I attached a wrong file.Here is the context file
This is the thread stack trace when i suspend in the debug mode.


Thread [task-scheduler-4] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.parkNanos(Object, long) line: 198
AbstractQueuedSynchronizer$ConditionObject.awaitNa nos(long) line: 2025
DelayQueue<E>.take() line: 164
ScheduledThreadPoolExecutor$DelayedWorkQueue.take( ) line: 609
ScheduledThreadPoolExecutor$DelayedWorkQueue.take( ) line: 602
ScheduledThreadPoolExecutor(ThreadPoolExecutor).ge tTask() line: 947
ThreadPoolExecutor$Worker.run() line: 907
Thread.run() line: 662

Thread [task-scheduler-10] (Suspended)
SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int) line: not available [native method]
SocketInputStream.read(byte[], int, int) line: 129
StreamDecoder.readBytes() line: 264
StreamDecoder.implRead(char[], int, int) line: 306
StreamDecoder.read(char[], int, int) line: 158
InputStreamReader.read(char[], int, int) line: 167
BufferedReader.fill() line: 136
BufferedReader.readLine(boolean) line: 299
BufferedReader.readLine() line: 362
FTPClient(FTP).__getReply() line: 294
FTPClient(FTP).getReply() line: 619
FTPClient.completePendingCommand() line: 1244
FTPClient.initiateListParsing(FTPFileEntryParser, String) line: 2309
FTPClient.initiateListParsing(String, String) line: 2269
FTPClient.listFiles(String) line: 2046
FtpSession.list(String) line: 62
FtpSession.list(String) line: 38
CachingSessionFactory$CachedSession.list(String) line: 121
FtpInboundFileSynchronizer(AbstractInboundFileSync hronizer<F>).synchronizeToLocalDirectory(File) line: 145
FtpInboundFileSynchronizingMessageSource(AbstractI nboundFileSynchronizingMessageSource<F>).receive() line: 146
SourcePollingChannelAdapter.receiveMessage() line: 111
SourcePollingChannelAdapter(AbstractTransactionSyn chronizingPollingEndpoint).doPoll() line: 67
AbstractPollingEndpoint$1.call() line: 146
AbstractPollingEndpoint$1.call() line: 144
AbstractPollingEndpoint$Poller$1.run() line: 236
ErrorHandlingTaskExecutor$1.run() line: 52
SyncTaskExecutor.execute(Runnable) line: 48
ErrorHandlingTaskExecutor.execute(Runnable) line: 49
AbstractPollingEndpoint$Poller.run() line: 231
ReschedulingRunnable(DelegatingErrorHandlingRunnab le).run() line: 51
ReschedulingRunnable.run() line: 81
Executors$RunnableAdapter<T>.call() line: 441
FutureTask$Sync.innerRun() line: 303
ScheduledThreadPoolExecutor$ScheduledFutureTask<V> (FutureTask<V>).run() line: 138
ScheduledThreadPoolExecutor$ScheduledFutureTask<V> .access$301(ScheduledThreadPoolExecutor$ScheduledF utureTask) line: 98
ScheduledThreadPoolExecutor$ScheduledFutureTask<V> .run() line: 206
ThreadPoolExecutor$Worker.runTask(Runnable) line: 886
ThreadPoolExecutor$Worker.run() line: 908
Thread.run() line: 662

[Gary Russell]

SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int) line: not available [native method]
SocketInputStream.read(byte[], int, int) line: 129
StreamDecoder.readBytes() line: 264
StreamDecoder.implRead(char[], int, int) line: 306
StreamDecoder.read(char[], int, int) line: 158
InputStreamReader.read(char[], int, int) line: 167
BufferedReader.fill() line: 136
BufferedReader.readLine(boolean) line: 299
BufferedReader.readLine() line: 362
FTPClient(FTP).__getReply() line: 294
FTPClient(FTP).getReply() line: 619
FTPClient.completePendingCommand() line: 1244
FTPClient.initiateListParsing(FTPFileEntryParser, String) line: 2309
FTPClient.initiateListParsing(String, String) line: 2269
FTPClient.listFiles(String) line: 2046
...

This thread dump simply shows that we are waiting for the ftp server to reply while we are trying to get a list of tiles - the problem is in the server.

[Kruti Shah]

Thank you so much. Yes,as you said all pollers are actually working fine.I think there is some other problem.

Currently,I have ovverriden the method postProcessClientBeforeConnect in my custom sessionFactory

protected void postProcessClientBeforeConnect(FTPClient ftpClient) throws IOException {
		 System.out.println("in before connect");
	     ftpClient.setDataTimeout(120000);
	     ftpClient.setDefaultTimeout(120000);
	  }

what configurations would I be required to do if I want to disconnect if no files are present on the server and try again after some time.This is required as the file with same name is generated after every 15 mins. and it may override my previous file if it is not detected at proper time.

Currently,once the read is timed out it does not automatically connect again and only when i manually disconnect it starts working again.And at the time of read timed out the 3 ftp inbound adapters thread wait at SocketOutputStream.socketWrite0(FileDescriptor,byt e[],int,int).I understand that the poller is waiting to write but because of read timed out it is not able to read.Pls. help me what needs to be done.

There is absolutely no problem with the 4th poller which puts the file from local to remote.

Looking forward to your Reply.Thank You!

[Kruti Shah]

The poller is at SocketInputStream.socketRead0(FileDescriptor,byte[],int,int) but it still does not detect the file on the server.This happens only if there are no files on server for around 10-15 mins and it gives this exception in between
Localized Message:Problem occurred while synchronizing remote to local directory
Cause:Connection refused: connect.(I have removed defaulttimeout and datatimeout,I use DefaultFTPSessionFactory)

It is only that 1 poller stops detecting though it is waiting on read,the other two poller detects files properly and processes it.

Currently,I have also used FTPClient instance in my inbound channel adapter filters inorder to rename the file on the server as soon as it is detected and disconnect the instance in the same.

[Kruti Shah]

This thread dump simply shows that we are waiting for the ftp server to reply while we are trying to get a list of tiles - the problem is in the server.

Thank u it is indeed a server problem.But in such a case ,these threads keep on reading when there is no proper response from the server,I need to manually disconnect and reconnect the ftp server for it to detect files again.I want to handle this suitation in a way that if there is no response from the server for x amount of time disconnect and then again try to connect after some time.Is setting timeout the solution?Which timeouts should be set?

Pls. advise.Thank you!

[Gary Russell]

Your existing code

ftpClient.setDataTimeout(120000);

should handle that situation.

[Kruti Shah]

Thank You so much!!Its working now