Results 1 to 2 of 2

Thread: multiple http-outbound gateway to download files

Hybrid View

  1. #1
    Join Date
    Jan 2013
    Posts
    6

    Default multiple http-outbound gateway to download files

    hi

    i have a question about multiple http outbound connection.
    in my use case i need to download files via http, i'm using this:
    PHP Code:
    <int:channel id="fileList"></int:channel>
    <
    int-http:outbound-gateway id="httpOutboundGateway"         
            
    url="{outboundGatewayUrl}" request-channel="fileList"
            
    reply-channel="channel13" http-method="GET"
            
    expected-response-type="byte[]" charset="UTF-8">
            <
    int-http:uri-variable name="outboundGatewayUrl" expression="payload" />
    </
    int-http:outbound-gateway>
    <
    int:channel id="channel13"></int:channel>
    <
    int-file:outbound-channel-adapter id="dwonloadChannel" channel="channel13"
            
    directory-expression="'${fileToProcess.dir}'" 
            
    filename-generator-expression="'file-'.concat(headers['jms_timestamp']).concat('.ready')"
            
    auto-create-directory="true" /> 
    this works fine (but if you have better option, i'd be happy to know )
    Note-the fileList is populated with msg' by a splitter, which may produce 10-60 links at once.
    My question is about, making this a multi-threaded process. meaning that i want to download ~5 files simultaneously.
    will it be enough to change the channel Q
    PHP Code:
    <int:channel id="fileList"> <int:queue capacity="5"/></int:channel
    and this?
    PHP Code:
    <int:channel id="channel13"> <int:queue capacity="5"/></int:channel
    or do i need to use custom task-execution (an example would be very helpful)?

    thanks!

  2. #2
    Join Date
    Mar 2010
    Location
    Gtr Philadelphia, PA
    Posts
    2,147

    Default

    It's probably better to use an ExecutorChannel to control the concurrency.

    http://static.springsource.org/sprin...ecutor-channel

    http://static.springsource.org/sprin...xecutorchannel

    You also might want to consider using (s)ftp instead of http, to avoid having to store the downloaded file in memory.

    If you have to use HTTP, you might want to open an improvement JIRA issue to allow the content to be written directly to disk...

    https://jira.springsource.org/browse/INT

    ... or, even better - implement the feature and submit a pull request...

    https://github.com/SpringSource/spri...ONTRIBUTING.md
    Gary P. Russell
    Spring Integration Team
    SpringSource, a division of VMware

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •