Framework performance after using it quite a while
Haven't found something similar so new post is here.
We have a problem when running Spring Batch jobs for a while. With initial state of database - all tables are empty - we take stop watch metrics and receive that Spring Batch does it's job in 7 seconds while whole jobs runs for about 2 minutes.
After that we launch ~30000 jobs and try to re-take the metrics from above. And now Spring Batch works for about 190 seconds and whole job is about 5 minutes.
Number of jobs is growing in about 2000 per week so performance downgrade become visible in 2 months of usage and in 5 months Spring Batch takes more time to perform own code than the job is run.
Any ideas what's the deal?
Our current plan is to override StepExecutionDao and ExecutionContextDao so they won't bother database with unnecessary saves while we don't use restartable jobs. How does this sound like?
Purge BATCH_* tables data
Does the Spring Batch framework itself provide any solution towards purging data from those Spring Batch tables?
It would be nice to have an out-of-box puring mechanism (thru jobs-context XML settings?).
Thanks & Regards,
Clelio de Souza