Broken Pipe After Sending Multiple Request to Outbound FTP Adapter

[sanspeaker]

Hi -

After sending the same request about 3 times to an outbound FTP adapter, I get the following error:

Caused by: org.springframework.integration.MessageDeliveryExc eption: Error handling message for file [/tmp/test.txt]
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:111)
at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:78)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :110)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:9 7)
at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:44)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:128)
at org.springframework.integration.core.MessagingTemp late.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemp late.send(MessagingTemplate.java:149)
at org.springframework.integration.router.AbstractMes sageRouter.handleMessageInternal(AbstractMessageRo uter.java:220)
at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:78)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :110)
... 17 more
Caused by: java.lang.IllegalStateException: failed to copy file
at org.springframework.integration.ftp.session.FtpSes sion.put(FtpSession.java:96)
at org.springframework.integration.file.remote.sessio n.CachingSessionFactory$CachedSession.put(CachingS essionFactory.java:132)
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.sendFileToRemoteD irectory(FileTransferringMessageHandler.java:181)
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:100)
... 28 more
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutp utStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStre am.java:136)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder. java:202)
at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.j ava:263)
at sun.nio.cs.StreamEncoder.write(StreamEncoder.java: 106)
at java.io.OutputStreamWriter.write(OutputStreamWrite r.java:190)
at java.io.BufferedWriter.flushBuffer(BufferedWriter. java:111)
at java.io.BufferedWriter.flush(BufferedWriter.java:2 35)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.jav a:472).

If I restart the server, it starts working again.

Here is my configuration:

<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- Ftp Client Factory -->
<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

<bean id="ftpClientFactory"
class="org.springframework.integration.ftp.session .DefaultFtpSessionFactory">
<property name="host" value="13.4.15.11"/>
<property name="port" value="21"/>
<property name="username" value="test"/>
<property name="password" value="test"/>
<property name="clientMode" value="0"/>
<property name="fileType" value="3"/>
</bean>

<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- Outbound FTP Adapter -->
<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<ftp:outbound-channel-adapter id="outboundFtpAdapter"
channel="outboundFtpChannel"
session-factory="ftpClientFactory"
charset="UTF-8"
remote-filename-generator-expression="headers['fileName']"
remote-directory="."/>

<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
<!-- FTP Channel -->
<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

<int:channel id="outboundFtpChannel">
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>

From the SI reference documentation, I read "the SessionFactory will perform validation to make sure that it never returns a stale session to the adapter. If a stale session was encountered, it will be removed from the pool, and a new one will be created."

Any ideas? Configuration issue? Please help.

I am connecting to a FileZilla FTP server. I am using SI 2.0.0.

Thanks,
Joseph

[oleg.zhurakousky]

You may be reading documentation for 2.0.1, but using 2.0.0.
The issue with the stale connection was fixed after 2.0.0 https://jira.springsource.org/browse/INT-1665

So i please upgrade to 2.0.1 and let us know if you still have this error.

[sanspeaker]

Thanks a lot for the quick reply.

I upgraded to 2.0.1.

I am now receiving the following error:


org.springframework.integration.dispatcher.Aggrega teMessageDeliveryException: All attempts to deliver Message to MessageHandlers failed. Multiple causes:
Failed to transfer file [/tmp/test.txt.tmp] from local working directory to remote FTP directory.
Failed to transfer file [/tmp/test.txt.tmp] from local working directory to remote FTP directory.
See below for the stacktrace of the first cause.
at org.springframework.integration.dispatcher.Unicast ingDispatcher.handleExceptions(UnicastingDispatche r.java:160)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :123)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:9 7)
at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:44)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.core.MessagingTemp late.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemp late.send(MessagingTemplate.java:149)
at org.springframework.integration.core.MessagingTemp late.convertAndSend(MessagingTemplate.java:189)
at org.springframework.integration.gateway.MessagingG atewaySupport.send(MessagingGatewaySupport.java:18 4)
at org.springframework.integration.jms.ChannelPublish ingJmsMessageListener.onMessage(ChannelPublishingJ msMessageListener.java:243)
at org.springframework.jms.listener.AbstractMessageLi stenerContainer.doInvokeListener(AbstractMessageLi stenerContainer.java:535)
at org.springframework.jms.listener.AbstractMessageLi stenerContainer.invokeListener(AbstractMessageList enerContainer.java:495)
at org.springframework.jms.listener.AbstractMessageLi stenerContainer.doExecuteListener(AbstractMessageL istenerContainer.java:467)
at org.springframework.jms.listener.AbstractPollingMe ssageListenerContainer.doReceiveAndExecute(Abstrac tPollingMessageListenerContainer.java:325)
at org.springframework.jms.listener.AbstractPollingMe ssageListenerContainer.receiveAndExecute(AbstractP ollingMessageListenerContainer.java:243)
at org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.invokeL istener(DefaultMessageListenerContainer.java:1058)
at org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.execute OngoingLoop(DefaultMessageListenerContainer.java:1 050)
at org.springframework.jms.listener.DefaultMessageLis tenerContainer$AsyncMessageListenerInvoker.run(Def aultMessageListenerContainer.java:947)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.springframework.integration.MessageDeliveryExc eption: Failed to transfer file [/tmp/m2.txt.tmp] from local working directory to remote FTP directory.
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:102)
at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:78)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :110)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:9 7)
at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:44)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:128)
at org.springframework.integration.core.MessagingTemp late.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemp late.send(MessagingTemplate.java:149)
at org.springframework.integration.router.AbstractMes sageRouter.handleMessageInternal(AbstractMessageRo uter.java:220)
at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:78)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :110)
... 17 more
Caused by: java.io.IOException: Failed to rename './test.txt.tmp.writing' to test.txt'. Server replied with: 553 file exists

at org.springframework.integration.ftp.session.FtpSes sion.rename(FtpSession.java:111)
at org.springframework.integration.file.remote.sessio n.CachingSessionFactory$CachedSession.rename(Cachi ngSessionFactory.java:149)
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.sendFileToRemoteD irectory(FileTransferringMessageHandler.java:170)
at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:95)
... 28 more

The FileZilla settings are set to overwrite existing files.

When I use the command line windows FTP program it overwrites successfully.

When I revert back to 2.0.0 SI jars, the file is successfully overwritten, but I get the same stale connection issue as mentioned earlier.

Is there a configuration to specify to overwrite exiting files on the SI side for this new version of the SI jars? This was working fine with 2.0.0 for me.

Please help.

Thanks,
Joseph

[oleg.zhurakousky]

Joseph

The rename functionality was added in 2.0.1. Before it was simply uploading the file under its final name which could present issues since the file might still be written. So now we are going through the same process on the upload as download where we writing file with .writing extension and when its fully written we rename it.

Now that is where your exception is happening. I did test it with Filezila, but i guess need to look at it a little deeper.

Could you please open a JIRA issue for it.

[sanspeaker]

No problem, how do I create a JIRA item? Do you have a link?

Thanks,
Joseph

[oleg.zhurakousky]

https://jira.springsource.org/

Create an account and fire away. The name of the project is Spring Integration. I can do it for you but if you do it you'll be notified automatically when its resolved.
Also when opening up link it to this forum (there'll be a special field for that)

[sanspeaker]

I created a JIRA ticket, INT-1742.

Any idea on the turn around time for fix that I can download for communication purposes to the project lead?

Thanks again for the quick response times, we appreciate it much,

Joseph

[oleg.zhurakousky]

It will be fixed today.
Are you able to test against the nightly snapshots?

btw, i was able to reproduce it, so that is always a good sign.

[sanspeaker]

You guys are so on top of things; I love the energy behind Spring.

Are there instructions on how I can build against the nightly snapshot? I have never done this before.

Thanks,
Joseph

[oleg.zhurakousky]

Are you using Maven?