dynamic inbound sftp and double messages into the queue.

[yarix]

hi,

my problem is with 2 inbound sftp (currently hard-coded) and a single outbound jms. in the configuration below i'm monitoring 2 different sftp sites, the files from both sites will be downloaded to a single local folder.

	<bean id="SftpRawFileFilter" class="my.app.SftpRawFileFilter"></bean>
	<int:message-history/>
	
	<bean id="ser1SftpFactory" class="my.app.CustomSftpSessionFactory">
		<property name="dbKey" value="ser1"/>
	</bean>	
	<int-sftp:inbound-channel-adapter id="ser1SftpInbondAdapter"
		session-factory="ser1SftpFactory" 
		remote-directory="/allFiles" charset="UTF-8"
		local-directory="file:/tmp/inboundSFTP"
		auto-create-local-directory="true" 
		delete-remote-files="false"
		filter="SftpRawFileFilter">
		<int:poller fixed-rate="20" time-unit="SECONDS" max-messages-per-poll="100"/>
	</int-sftp:inbound-channel-adapter>
	<int:header-enricher input-channel="ser1SftpInbondAdapter"
		output-channel="fromSftp" >
		<int:header name="app.ChannelId" ref="ser1SftpFactory" method="connectionIdHeader" />
		<int:header name="app.PartnerId" ref="ser1SftpFactory" method="getPartner" />
		<int:header name="app.FileExtenssion" ref="ser1SftpFactory" method="fileExtnessionHeader" />
	</int:header-enricher>

	<bean id="ser2SftpFactory" class="my.app.CustomSftpSessionFactory">
		<property name="dbKey" value="p2"/>
	</bean>	
	<int-sftp:inbound-channel-adapter id="ser2SftpInbondAdapter"
		session-factory="ser2SftpFactory"
		remote-directory="/Downloads/" charset="UTF-8"
		local-directory="file:/tmp/inboundSFTP"
		auto-create-local-directory="true" 
		delete-remote-files="false"
		filter="SftpRawFileFilter">
		<int:poller fixed-rate="20" time-unit="SECONDS" max-messages-per-poll="100"/>
	</int-sftp:inbound-channel-adapter>
	
	<int:header-enricher input-channel="ser2SftpInbondAdapter"
		output-channel="fromSftp" >
		<int:header name="app.ChannelId" ref="ser2SftpFactory" method="connectionIdHeader" />
		<int:header name="app.PartnerId" ref="ser2SftpFactory" method="getPartner" />
		<int:header name="app.FileExtenssion" ref="ser2SftpFactory" method="fileExtnessionHeader" />
	</int:header-enricher>
	
	<int:channel id="fromSftp" datatype="java.io.File">
		<int:queue />
		<int:interceptors>
			<ref bean="auditInterceptor"/>
			<int:wire-tap channel="loggingCh"/>
		</int:interceptors>
	</int:channel>

  	<int-jms:outbound-channel-adapter id="jmsFiles"
		destination-name="app.Files" channel="fromSftp" /> 

	<int:logging-channel-adapter id="loggingCh"		
		level="WARN" logger-name="my.app.loggingCh"
		log-full-message="true"/>

• My expectation is to get a single JMS message for every file that is on the local folder, but actually i get 2 messages for every file. i've added message-history and i see that i get a message per sftp-channel instead of per file. Why is that happening?
• and, is there a nicer way to enrich the headers with the actual sftp customer? now i'm using my customized session factory and
  

  <int:header name="app.PartnerId" ref="ser2SftpFactory" method="getPartner" />

  i don't like the fact that i need to use different beans (ser1SftpFactory, ser2SftpFactory). it could be nicer if i was able to do use the header enricher from the inbound-sftp, and use SpEL like: #this.sessionFactory.partnerId

thanks!

[Gary Russell]

When using the inbound adapter, you can't use the same local directory.

They way these adapters work is to synchronize the remote and local directories (by fetching any files missing from the local dir), and then a separate process (within the adapter) generates a message for any new files found in the download directory; hence you are seeing two messages.

You need to use different local dirs or, you might want to consider using outbound-gateways instead; then you can "list", split and "get" (or "mget") the files, and there will be no crosstalk between the gateways.

Take a look at the gateway example in the sftp sample (it does ls, get, rm on the remote files)... https://github.com/SpringSource/spri...ter/basic/sftp

There is not currently a way to provide a header mapper (to generate custom headers) on the (s)ftp endpoints, but I can see some value in that; feel free to open a new feature JIRA issue https://jira.springsource.org/browse/INT, but I think you are stuck with your current technique for now.

[yarix]

Thank you Gary for the explanation. I've used 2 different local dirs.
I've also opened a jira - INT-2949 for improvement.