Spring Batch and Hadoop or GPars

[Jabberz]

I'm very interested to hear about the future of Spring Batch and if you will consider pluggable asychronous systems into the batch framework, so you could do things like use GPars (for us groovy fans) or Hadoop for large scale, concurrent batch processing. I thought I read somewhere that Hadoop was on the list.

[nitin.tripathi]

yes indeed, I'll be interested with Hadoop integration can you provide those references? what are the possibilites?

[Dave Syer]

Hadoop is not what I would call "pluggable", but we have made a start on Spring Hadoop (http://github.com/SpringSource/spring-hadoop). It will provide a platform for Spring Batch integration eventually (actually nearly there now, but haven't had time to do much on that project recently), and is already showing promise as a way to make Hadoop programming more bearable (Spring-like).

We're also open to ideas about GPars, Scala etc. I'm not really sure what that would look like right now though. Someone should try it and see what happens.