I jumped the gun and opened a JIRA ticket for this problem, but I wanted to also run it by folks on the forum:
https://jira.springsource.org/browse/INT-2048

It appears that as the number of files in a remote directory grows, the performance of the ensureDirectoryExists method of FileTransferringMessageHandler will degrade. Has anyone else run into this issue? Have you found any ways to mitigate this?