What is file:inbound-channel-adapter max size limit?

[AnthonyRyan]

Hi all,

I'm new to Spring Integration and I was wondering what the max file size for an inbound-channel-adapter is. Can take it large files (we're talking 1-4 gigs)?

I'm hoping to poll a directory, read in a single file of many records, then split the records using a splitter and rout them to many different queues.

I think the design is ok so far but the thing that might kill it is the file size, which may mean I'll have to look at Batch instead.

Thanks for any help!

[Marten Deinum]

You can read files that large, if it is smart is another question... It generates a payload of 4 gigs...

Why not simply use both... Use Spring Integration to poll, get the filename (not the file content) and use that to kick-off spring batch. Basically best-of both worlds.

[AnthonyRyan]

Thanks for the quick reply Marten! I thought that maybe Batch might be the way to go alright... good idea of getting the file name and not the payload to kick off the batch job too.

Can I ask then, would the batch reader take the place of the splitter in the above scenario? And then I would have my proposed router in a second step, followed by other normal Integration components reading from these queues outside of the job (and possibly while the job is still crunching away)?

Cheers,
Tony

[Marten Deinum]

batch would read/split the file using an ItemReader and an ItemWriter could write those items to a queue/channel...

Big advantage is that you have restart/continue capabilities and don't have to have everything in memory (ofcourse instead of using Spring Batch you could also create your own ServiceActivator which streamingly reads from the file and pushes it to a channel).

[AnthonyRyan]

Well I guess if that's what Batch was made for I should really go down that route and not try to shoe-horn it into a pure Integration solution. And I suppose after the ItemWriter runs I can have a regular router running off that. Thanks for your help Marten!

[Mark Fisher]

If you grab the spring-batch-integration module that is part of spring-batch-admin, it has a few good components for combining the two projects. For example, there is an ItemWriter implementation that publishes Messages to a Spring Integration channel. That's a nice way to pass along to the rest of the flow from within the Batch job. Of course, it would not be too difficult to implement that writer yourself as well.

FWIW, we are planning to support these combined use-cases "out of the box" in the future. We're working on that now.

[AnthonyRyan]

That sounds like exactly the type of ItemWriter I'm going to need Mark, cheers. Is there a similar ItemReader implementation?

As described by Marten above, I'm gonna place a filename as a Message<String> on a regular Spring Integration channel and use that to hopefully kick of a Batch job?

[Mark Fisher]

If you look in the same spring-batch-integration module, you will find a Spring Integration MessageHandler that launches Batch jobs via a JobLauncher. The Message content is interpreted as JobParameters.

Have a look in spring-batch-admin and let me know if it's still unclear: http://static.springsource.org/spring-batch-admin/

Perhaps the best starting point is actually javadoc (see the 'org.springframework.batch.integration.*' packages): http://static.springsource.org/sprin...ocs/index.html

HTH,
Mark

[thapa25]

I don"t know about inbound-channel-adapter -max-size limit. I am also looking the same spring integration module. I need help from
anyone. I offer a business plan for you. For more.....