Generic Slave for Remote chunking

[fjcarretero]

Hi all,

I'm trying to design a batch architecture based on spring-batch and JMS. I have read a lot about remote chunking and I have a question.

I've seen that the slave that executes de ItemProcessor needs to create an specific context for each job. If you don't know the job beforehand, is there a way to create a generic slave, and along with the data pass the job definition (or the specific part of it)?

TIA
Felipe

[fjcarretero]

I have been investigating how to implement a generic slave. As I want the slave to be generic, the ChunkHandler shouldn't be initialized with a preconfigured ChunkProcessor. The ChunkProcessor must be passed to the ChunkHandler.

I have thought to pass the ChunkProcessor within the JMS message. To do this, ChunkProcessor must be Serializable.

One thing that I don't know is if this overhead of unmarshalling the ChunkProcessor is going to be expensive.

Is this a sensible way to go?

TIA
Felipe

[fjcarretero]

I have seen that GridGain has done something similar:

SerializableChunkProcessor

TIA
Felipe

[fjcarretero]

Hi all,

Finally I have make it work. I did 3 things:
- make all the objects serializable, specially the one's inside the ChunkProcessor
- use annotations for the custom objects
- enable loadtime-weaving in the remote slave to reconstruct the annotated objects

If I can, I will upload the code as an example

Thanks
Felipe

[ssuresh83]

do you know same way how to executed partitioned step remotely.

[fjcarretero]

Take a look at this project. There are plenty of examples.

http://code.google.com/p/springbatch-in-action/

Also is worth having a look to the different integrations with grids.

https://github.com/dsyer/spring-batch-grid

I have done my example with JMS (JBoss). How are you planning to pass the data to the slaves?

Regads,
Felipe

[ssuresh83]

I have followed the dave syers example , i am not passing data to remote nodes. The data is there in the database. It picks from database and executes.But i have problem running remote step it says with following error

step executions for given job execution are expected to be already saved

and i have posted in this forum.
http://forum.springsource.org/showth...ed-ERROR-GRIDG