SFTP Adapter Channel not being closed

[ramalingam]

I have the following Spring Configuration

  <context:property-placeholder location="/spring/ftp/ftp.properties"/>

	<bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
			<property name="host" value="${host}"></property>
			<property  name="user" value="${username}"></property>
			<property name="password" value="${password}"></property>
	</bean>

  <int-sftp:inbound-channel-adapter id="ftpInbound"
  								  	channel="ftpChannel"
  								  	auto-startup="true"
  								  	session-factory="sftpSessionFactory"
                                    remote-directory="${sftp.remoteDir}"
                                    auto-create-local-directory="true"
                                    delete-remote-files="false"
                                    local-directory="file:output"
                                    filter="fileModifiedFilter">
    <int:poller fixed-rate="60000"/>
  </int-sftp:inbound-channel-adapter>

  <int:channel id="ftpChannel">
    <int:queue/>
  </int:channel>
	<bean id="fileModifiedFilter" class="com.mycompany.AcceptModifiedFileFilter"></bean>

public class AcceptModifiedFileFilter<F> extends AbstractFileListFilter<F> {
	/**
	 * The logging object
	 */
	private static final Logger log = LoggerFactory.getLogger(AcceptModifiedFileFilter.class);

	private final Map<String, Integer> fileModTime = new HashMap<String, Integer>();

	private final Object monitor = new Object();

	public AcceptModifiedFileFilter() {
		// TODO Auto-generated constructor stub
	}


	public boolean accept(F file) {
		
		synchronized (this.monitor) {
			if (this.fileModTime.get(file.toString())==null) {
				fileModTime.put(file.toString(), ((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime());
				log.debug("First Instance of File" + " :: " + fileModTime  );
				return true;
			}
			if (this.fileModTime.get(file.toString()).compareTo(((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime()) == 0) {
				log.debug("File already downloaded" + " :: " + fileModTime );
				return false;
				}
			if (this.fileModTime.get(file.toString()).compareTo(((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime()) < 0) {
				// file was modified, update timestamp
				log.debug("File Modified & new file is downloaded now" + " :: " + fileModTime);
				fileModTime.put(file.toString(), ((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime());
				return true;
				}
				// modified in past
			log.debug("File was modified in the past" + " :: " + fileModTime);
				return false;	
			}
			
		}
}

---------------

Implemented the Filter Solution -- now still I dont have the local files updated
-------------------------------------Logoutput

02:41:11.274 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966}
02:41:11.277 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398}
02:41:11.277 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:41:11.278 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:41:11.278 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:43:05.240 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:44:04.915 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:45:07.401 [task-scheduler-7] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - File already downloaded :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
02:46:13.383 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:15 .=1348348533, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}

Even if I see first Instance of the file the local repository is not being overwritten with the new file

Is there any reason to this.
Regards,
Rams

[Gary Russell]

The inbound adapter is not designed to do this; even if you get the synchronizer to work, the FileReadingMessageSource used to read the local directory has an AcceptOnceFileListFilter baked into it, which only uses the file name to decide whether or not a file has already been processed. So, replacing the file will not result in a new message.

Consider using outbound gateways instead.

http://static.springsource.org/sprin...tbound-gateway

That way, you can 'ls' the remote directory, look at the file timestamp(s) and use 'get' to retrieve the file(s).

There's an example showing how to use the gateway in the sample apps; you'd just need to add a filter to ignore files where the timestamp hasn't changed.

[ramalingam]

Thanx a lot it did help me to better understand what inbound adaptors are for.

The inbound adapter is not designed to do this; even if you get the synchronizer to work, the FileReadingMessageSource used to read the local directory has an AcceptOnceFileListFilter baked into it, which only uses the file name to decide whether or not a file has already been processed. So, replacing the file will not result in a new message.

Consider using outbound gateways instead.

http://static.springsource.org/sprin...tbound-gateway

That way, you can 'ls' the remote directory, look at the file timestamp(s) and use 'get' to retrieve the file(s).

There's an example showing how to use the gateway in the sample apps; you'd just need to add a filter to ignore files where the timestamp hasn't changed.