Results 1 to 4 of 4

Thread: 2.0.4.RELEASE - make sure your SFTP directories have a trailing slash!

  1. #1

    Default 2.0.4.RELEASE - make sure your SFTP directories have a trailing slash!

    It seems like something has changed between 2.0.3 and 2.0.4 where if you previously had an SFTP inbound gateway with a remote directory configured, and it did not have a trailing slash, you might start seeing the following exception after upgrading:

    Code:
    org.springframework.core.NestedIOException: failed to read file; nested exception is 2: No such file
    	at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:100)
    	at org.springframework.integration.file.remote.session.CachingSessionFactory$CachedSession.read(CachingSessionFactory.java:137)
    	at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.copyFileToLocalDirectory(AbstractInboundFileSynchronizer.java:176)
    	at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:138)
    	at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:144)
    	at org.springframework.integration.endpoint.SourcePollingChannelAdapter.doPoll(SourcePollingChannelAdapter.java:89)
    	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
    	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:144)
    	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:207)
    	at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
    	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
    	at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
    	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:202)
    	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51)
    	at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    	at java.lang.Thread.run(Thread.java:680)
    Caused by: 2: No such file
    	at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
    	at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1741)
    	at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1011)
    	at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:986)
    	at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:96)
    	... 22 more
    I haven't tracked it down to any specific code change in 2.0.4.RELEASE, but my project definitely works in 2.0.3 and generates the above error with 2.0.4. Adding a trailing slash to the remote directory resolves the issue.

  2. #2
    Join Date
    Jan 2008
    Location
    Mohnton, PA USA (that's near Philadelphia)
    Posts
    2,148

    Default

    This is actually strange
    We did do some work here for 2.0.4 but it mainly involved exposing remoteFileSeparator for configuration since we got some issues related to fileSeparator character specific to the platform. However, the default value remains the same '/' and it was never appended at the end;
    For example
    Code:
    2.0.3 AbstractInboundFileSynchronizer
    private String remoteFileSeparator = "/";
    . . .
    private void copyFileToLocalDirectory(String remoteDirectoryPath, F remoteFile, File localDirectory, Session session) throws IOException {
    		String remoteFileName = this.getFilename(remoteFile);
    		String remoteFilePath = remoteDirectoryPath + remoteFileSeparator + remoteFileName;
    ...
    }
    
    2.0.4 AbstractInboundFileSynchronizer
    private String remoteFileSeparator = "/";
    . . .
    private void copyFileToLocalDirectory(String remoteDirectoryPath, F remoteFile, File localDirectory, Session session) throws IOException {
    		String remoteFileName = this.getFilename(remoteFile);
    		String remoteFilePath = remoteDirectoryPath + remoteFileSeparator + remoteFileName;
    ...
    }
    and i can't see any changes to the read method of SftpSession between 2.0.3 and 2.0.4

    Come to think of it, what version of JSCH is on your classpath?
    Also, could you provide more info about your client/server environment so I can try to reproduce it here.

  3. #3

    Default

    I thought of JSCH too, but it hasn't changed between the releases. I'm just pulling it in as a transitive dependency:

    Code:
    [INFO] +- org.springframework.integration:spring-integration-sftp:jar:2.0.4.RELEASE:compile
    [INFO] |  +- com.jcraft:jsch:jar:0.1.42:compile
    [INFO] |  +- org.springframework.integration:spring-integration-stream:jar:2.0.4.RELEASE:compile
    [INFO] |  \- org.springframework:spring-context-support:jar:3.0.5.RELEASE:compile
    
    [INFO] +- org.springframework.integration:spring-integration-sftp:jar:2.0.3.RELEASE:compile
    [INFO] |  +- com.jcraft:jsch:jar:0.1.42:compile
    [INFO] |  +- org.springframework.integration:spring-integration-stream:jar:2.0.3.RELEASE:compile
    [INFO] |  \- org.springframework:spring-context-support:jar:3.0.5.RELEASE:compile
    So I stepped through org.springframework.integration.file.remote.synchr onizer.AbstractInboundFileSynchronizer#copyFileToL ocalDirectory and remoteFilePath definitely did not contain a slash in the line:

    Code:
    				session.read(remoteFilePath, fileOutputStream);
    I'll give you some more details in a bit. Here's what my SFTP config looks like:

    Code:
        <sftp:inbound-channel-adapter
                id="sftpInboundAdapter"
                session-factory="sftpSessionFactory"
                cache-sessions="true"
                remote-directory="${sftp.dir}"
                filename-regex="prefix.*\.txt"
                delete-remote-files="true"
                local-directory="#{ sftpDirectory.absolutePath }"
                auto-create-local-directory="true"
                channel="convertIncomingFiles">
            <integration:poller fixed-delay="5000" error-channel="errorChannel"/>
        </sftp:inbound-channel-adapter>
    The sftpDirectory is a java.util.File instance created in the Spring beans context, and sftp.dir is resolved by a PropertyPlaceholderConfigurer.

  4. #4

    Default

    It seems to be related to "INT-1885 added support for empty remote-file-separator to the FTP Inbound Channel Adapter" - this fragment in org.springframework.integration.file.config.Abstra ctRemoteFileInboundChannelAdapterParser#parseSourc e:

    Code:
    		String remoteFileSeparator = element.getAttribute("remote-file-separator");
    		if (remoteFileSeparator != null){
    			synchronizerBuilder.addPropertyValue("remoteFileSeparator", remoteFileSeparator);
    		}
    Initializes the separator to empty unless you explicitly provide a value in the XML config. So the below works fine in 2.0.3, but not in 2.0.4:

    Code:
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:sftp="http://www.springframework.org/schema/integration/sftp"
           xmlns:integration="http://www.springframework.org/schema/integration"
           xmlns:stream="http://www.springframework.org/schema/integration/stream"
           xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
               http://www.springframework.org/schema/integration/sftp http://www.springframework.org/schema/integration/sftp/spring-integration-sftp-2.0.xsd
               http://www.springframework.org/schema/integration/stream http://www.springframework.org/schema/integration/stream/spring-integration-stream-2.0.xsd">
    
        <sftp:inbound-channel-adapter
                id="sftpInboundAdapter"
                session-factory="sftpSessionFactory"
                cache-sessions="true"
                remote-directory="input"
                filename-regex="test.*\.txt"
                delete-remote-files="true"
                local-directory="/tmp/files"
                auto-create-local-directory="true"
                channel="files">
            <integration:poller fixed-delay="5000" error-channel="errorChannel"/>
        </sftp:inbound-channel-adapter>
    
        <bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
            <property name="host" value="vm"/>
            <property name="port" value="22"/>
            <property name="user" value=".."/>
            <property name="password" value="..."/>
        </bean>
    
        <integration:channel id="files"/>
    
        <integration:transformer
                input-channel="files" output-channel="fileNames"
                expression="'File downloaded: ' + payload.name + '.'"/>
    
        <integration:channel id="fileNames"/>
    
        <stream:stderr-channel-adapter append-newline="true" channel="fileNames"/>
    
    </beans>
    Changing the SFTP inbound adapter by adding an explicit remote-file-separator setting like so fixes it again:

    Code:
        <sftp:inbound-channel-adapter
                id="sftpInboundAdapter"
                session-factory="sftpSessionFactory"
                cache-sessions="true"
                remote-directory="input"
                remote-file-separator="/"
                filename-regex="test.*\.txt"
                delete-remote-files="true"
                local-directory="/tmp/files"
                auto-create-local-directory="true"
                channel="files">
            <integration:poller fixed-delay="5000" error-channel="errorChannel"/>
        </sftp:inbound-channel-adapter>
    The cause for all this is that element.getAttribute("remote-file-separator") returns an empty string even if the XML element has no such attribute, so changing it like so will fix the default behavior back to what it was.

    The solution is to change org.springframework.integration.file.config.Abstra ctRemoteFileInboundChannelAdapterParser#parseSourc e like so:

    Code:
    		if (element.hasAttribute("remote-file-separator")){
    			synchronizerBuilder.addPropertyValue("remoteFileSeparator", element.getAttribute("remote-file-separator"));
    		}
    I've opened an issue and attached a patch with unit tests in JIRA - INT-1921.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •