Hibernate Flushing in ItemWriter

[litius]

Have a question that relates to Hibernate flushing (JIRA: BATCH-194)

We have our own persistence framework here that is currently under revision of adding a client's flush flag to enable an auto flush ("hibernateTemplate.flush()") on any CRUD (well CUD really).

But for now we need the Hibernate Template to flush after each item is processed (or "written", as they say now days ). Simple reason would be a "DataIntegrityViolationException" that is thrown outside of the ItemWriter in ItemOrientedStep:

try {
    itemHandler.mark();                      //     should not mark() be called after commit, in case if commit fails? :-/
    transactionManager.commit(transaction);  // <-- exception here, since there was no flush
}
catch (Exception e) {
    fatalException.setException(e);
    stepExecution.setStatus(BatchStatus.UNKNOWN);
    logger.error("Fatal error detected during commit.");
    throw new FatalException("Fatal error detected during commit", e);
}

and therefore is not intercepted by "public void onWriteError(Exception ex, Object item)".

I know that you guys have a "HibernateAwareItemWriter" (from samples):

       <!-- This is a framework class that needs a delegate and also needs to be registered as a RepeatInterceptor in the chunk -->
       <bean id="hibernateItemWriter"
           class="org.springframework.batch.item.database.HibernateAwareItemWriter">
           <property name="sessionFactory" ref="sessionFactory" />
           <property name="delegate" ref="hibernateCreditWriter" />
       </bean>

But we already have existing ItemWriters that we would not want to re-write, or wrap all of them in "HibernateAwareItemWriter".
Any other current solution available in 1.0.1 (besides AOP)?

( What we need is simple: to catch any exceptions that are caused in ItemWriter by "public void onWriteError" method, so the item can be skipped, or acted upon exception accordingly )

Thank you.

[lucasward]

There's quite a few questions here, but I'll try and break it down as best I can:

1) Mark called before commit. I understand your point, but if a commit fails, the job will fail, so it's a moot point really. It wouldn't make too much difference whether it was called before or after commit.

2) Hibernate flushing. The HibernateAwareItemWriter will fail the first time without calling onWriteError, as you correctly noted. However, after that it will commit after every record, which should lead to an exception in onWriteError. However, now that I think about it the Hibernate flush even in that scenario might be on itemwriter flush, which wouldn't be caught in the 'onWriterError' I'll have to double check this.

I'm not a Hibernate expert by any means, but isn't there a way to configure Hibernate to flush eagerly? Or, couldn't you create your own wrapper that simply delegated to a Hibernate dao and called flush itself after each write to the delegate?

[litius]

Lucas,

Thanks for taking time...

1. Mark called before commit. Just noticed it while browsing the framework code. Thought that it is better to leave this up to the client to decide on whether to consider a "commit failure" as FATAL. If it is not FATAL, then it would make sense to call mark() after commit. One of examples of a "none fatal commit":

      -> processing items
      -> ODS went down
      -> commit failed
      -> current item(s - depending on commit interval) send to workflow (queue for manual intervention)
      -> ODS went up
      -> following items of the same step/job are all processed

(just a thought - just seems to be more of a "client decision" than a framework hard-coded one.)

2. As to Hibernate flushing - that is exactly what we are waiting on happening "configure Hibernate to flush eagerly" from our persistence framework guys, but just did not want it to be a stopper to test our Spring Batch listeners, therefore we're curious about a temp solution

Thanks again.

[Dave Syer]

I don't really understand the aversion to HibernateAwareItemWriter, since it is designed specifically to address this issue. Calling Session.flush() after every item might be a good idea for you (in which case feel free to add an ItemWriteListener that does it), but it isn't in general because the perormance benefit of batching output on flush are significant.

[litius]

Dave,

No aversion We have several hibernatish ItemWriters, and every one should be wrapped with HibernateAwareItemWriter which is a slight over kill for a temp solution. As to ItemWriteListener, it has to have an access to hibernate session which, sort of, "breaks an abstraction"...

We do have a persistence framework where all repositories (DAOs) come from, and which will have a "flush mode" flag, that we'll be able to use, if we need an eager flushing.

All our Repositories (DAOs) extend HibernateDaoSupport, so for now we just inject our custom HiberbnateTemplate with earger flushing into it from the batch side of the universe.

Thanks for your thoughts.