Mar 6th, 2013, 08:02 AM
Hi, I'm new to spring-batch and I don't the best way to do this.
My batch application generates some files to be downloaded later by the client, and I do all the processing in a class that I implemented org.springframework.batch.core.step.tasklet.Taskle t.
I need to make my application multi-thread, but the amount of thread to run each job instance will be defined by the client. So I'll have a pool with, say, 10 threads, and the client in his request will say how many threads he wants for his file generation.
What is the best way to accomplish this? I've already read the Chapter 7 form the spring-batch reference, but I still have no idea.
Mar 6th, 2013, 09:57 AM
I still hope to find a better solution, but it works - at least for me :-) I spent some time trying to use late binding for the property "concurrencyLimit" of the bean "taskExecutor", but I finally gave up.
<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTa skExecutor"/>
<batch:tasklet task-executor="taskExecutor" throttle-limit="10">
<batch:chunk reader="itemReader" writer="itemWriter" commit-interval="1"/>
I start the job like that:
final int concurrencyLimit = 7; // replace this value with whatever you like
ApplicationContext ctx = new ClassPathXmlApplicationContext("spring/test-config.xml");
JobLauncher launcher = (JobLauncher) ctx.getBean("jobLauncher");
SimpleAsyncTaskExecutor taskExecutor = (SimpleAsyncTaskExecutor) ctx.getBean("taskExecutor");
JobParameters jobParams = new JobParametersBuilder().toJobParameters();
Job job = (Job) ctx.getBean("myJob", Job.class);
Mar 6th, 2013, 10:23 AM
Thank you for the detailed answer!!
Originally Posted by BjRi
That already helped me a lot.
I see here that the setConcurrencyLimit will "set the maximum number of parallel accesses allowed".
Now, what if I want, in your example, to use exactly 7 threads, instead of using at most 7 threads? I would have to wait other jobs finish their execution... is it possible?
Mar 6th, 2013, 10:47 AM
Just to get it right:
You have a pool of 10 threads and there is currently one job running with 7 threads. If another job should be started with 4 threads, then the job with 7 threads has to finish first, so that there are four threads available for the new job?
That would be a tough task :-)
Mar 6th, 2013, 10:55 AM
Originally Posted by BjRi
Well, if anyone has any idea it is most welcome!
Mar 7th, 2013, 12:53 AM
multithreading can come handy here, for java , each spring batch job is a process , if you use thread pool executor , which is with JDK 1.6 or further you can , execute 10 or more jobs in join and they will wait for resources.
Tags for this Thread