Deleting historic Job related data through Spring Batch API

[Jakub.Kahovec]

Is there a way how to delete historic Job related data through API ? It seems to me that there is not, so is it something what is planned to be added ?

Thanks

Jakub

[Dave Syer]

It is supported in JobRepositoryTestUtils (i.e. not intended for production use, but could be used I guess judiciously). In general I do not like the idea of simply deleting management data. I'm sure there is a JIRA issue for archive features (probably marked as "won't fix"), but that gets to be quite complicated to provide a full service as part of the framework. Some sample jobs that did repository data management might be welcome, if anyone wants to contribute them.

[Jakub.Kahovec]

The reason I'm asking is when having JobRepository as an in-mem DB, one might be running out of memory soon. Currently I'm cleaning the completed jobs and their related data in a background thread using JDBC template. I just thought it'd be handy to have this functionality as part of API.

[arno]

I agree this would be handy, but perhaps its place should me more in Spring Batch Admin than in Spring Batch itself. Just a thought.

[mpat]

I also need to delete the Metadata tables that has data older than 90 days. Either through spring API or stored procedure. If you know how please reply. Thanks

[Miko95]

I also use an in-memory database for Spring Batch meta data and i need to clean this database periodically when the application is running (I use a background thread). One question i have is why the foreign key defined between Spring Batch tables doesn't specify ON DELETE CASCADE in order to simplify data deletion?
Another question is how to delete data stored in the JOB_INSTANCE table? I can delete all other data by deleting Job executions according to some columns of the JOB_EXECUTIONS table, but I don't have such columns in the JOB_INSTANCE table to determine if this instance corresponds to an already executed job.

I precise that I don't use the features of Spring Batch like restartability,...

Thanks

[ticino]

It is supported in JobRepositoryTestUtils (i.e. not intended for production use, but could be used I guess judiciously).

Hi, I tried to use JobRepositoryTestUtils to drop all jobs metadata older than x days. Somehow when passing list of job execution jobsinstance entry is delete using job_id for instance_id. Wrong instance are dropped. Another problem is that Instance is dropped even if other executions of this istance is still avariable.

here is the code i think is wrong:

public void removeJobExecutions(Collection<JobExecution> list) throws DataAccessException {
[...]
 for (JobExecution jobExecution : list) {
        jdbcTemplate.update(getQuery("delete from %PREFIX%JOB_PARAMS where JOB_INSTANCE_ID=?"),
            jobExecution.getId());
        jdbcTemplate.update(getQuery("delete from %PREFIX%JOB_INSTANCE where JOB_INSTANCE_ID=?"),
            jobExecution.getId());

i would propose something like :

 for (JobExecution jobExecution : list) {
      int numberJobExecutions =
          jdbcTemplate.queryForInt(getQuery("select count(JOB_EXECUTION_ID) from %PREFIX%JOB_EXECUTION where JOB_INSTANCE_ID=?"), jobExecution.getJobInstance().getId());
      if (numberJobExecutions == 0) {
        jdbcTemplate.update(getQuery("delete from %PREFIX%JOB_PARAMS where JOB_INSTANCE_ID=?"),
            jobExecution.getJobInstance().getId());
        jdbcTemplate.update(getQuery("delete from %PREFIX%JOB_INSTANCE where JOB_INSTANCE_ID=?"),
            jobExecution.getJobInstance().getId());
      }