Thread Safety While Running Multiple Instances Of The Same Job

[Dave Syer]

Interesting. I was aware of this issue but no-one raised it before - 2.0 will have a different listrener / writer contract so it will be easier there. What you have to do for now is store up the items (if multithreaded use a transaction resource like in the BatchSqlUpdateItemWriter) in an ItemReadListener, and then flush them in one query in a ChunkListener. Remember you can write one class that implements bith interfaces.

[Kmisaal]

Hey Dave.

I tried one scenario like this.

As I already told I am using processed flag.
Before starting any job I partition the source table data into 3 parts.
I set the value of processed flag = 'JOB1' for first part processed = 'JOB2' for second part and processed = 'JOB3' for the third part. Each part contain say 10K records. My chunk size is 100.

Now I run the 3 instances of my job passing the job parameter as job=JOB1 for first instance job = JOB2 for second instance and job = JOB3 for third instance.
In the reader query I use this passed job parameter in where clause as

select * from table1 where processed = < job parameter value >

This will ensure me that each job instance is working on logically separate data set. I run these 3 instances of the same job concurrently. I have only one job configuration file. Each job will set processed=1 once it successfully process the row.

This works good and shows good performance gain. Up to this point everything is good good hmm ;-)

The problem I faced when some instance fails other instances status is not getting correctly updated into spring batch tables. Even after 2 instances complete and one failed. Status for Completed instance is shown "Started" in the table.

I hope you got my scenario and question. Thank you for being patient and reading it completely. Please tell me am I doing the right thing ?

[Dave Syer]

That's pretty cool actually. Nice work.

I don't understand the question though. You are running three jobs concurrently. What is it that goes wrong? You are interrupting one of them, and the others are being affected. Are you sure they are not sharing state? Set them up each with a separate ApplicationCOntext to be on the safe side (e.g. use ClassPathXmlApplicationContextJobFactory).