Lucas, thank you for your answer.
I'll try to describe my use case as far as I can.
We ported our code to 1.0.0.FINAL.
We have a process that listens to a socket and receives data from a sending process. The sending process (it is in a different JVM) accumulates data from several thousand sensors.
Our listener parses the received data and writes it to 14 different files (as per sensor type). Then 14 jobs read these files, every job knows what files to read (according to file name pattern). For every sensor type there is a corresponding table in the database, plus one table that represents sensors hierarchy.
All 14 tables have a foreign key constraint on the hierarchy table.
And now the main part.
A job reads a file and tries to insert the data to the corresponding table.
When the job inserts data from a sensor that is not present in the hierarchy table, and fails on the foreign key constraint. In the RetryListener.onError() there is enough information for it to add the new sensor into the hierarchy and then retry the insert. The sensor addition to the hierarchy table is performed via a Hibernate Dao that is wrapped in a service, so that the service is defined with @Transactional(propagation=Propagation.REQUIRES_NE W). All the data insertion to the child table is performed via iBatis.
My problem is in the Hibernate call to the service. It is stuck as the surrounding iBatis transaction locks the table (we use MySQL InnoDb for all the tables).
We thought that by changing transaction isolation level on the spring-batch transactions (that run iBatis Daos) we will be able to eliminate the lock.
Also we use JpaTransactionManager.
As a workaround I used a different thread for the hierarchy table update, and that worked, but it's a partial and ugly solution, that may fail, I know.
Here is my RetryListener code
Code:
public class StepRetryListener extends RetryListenerSupport {
private ReporterService reporterService;
private ReporterType reporterType;
@Autowired
public void setReporterService(ReporterService reporterService) {
this.reporterService = reporterService;
}
public void setReporterType(ReporterType reporterType) {
this.reporterType = reporterType;
}
@Override
public void onError(RetryContext context, RetryCallback callback, Throwable throwable) {
Object data = ((ItemWriterRetryCallback)callback).getItem();
String errorMessage = throwable.getMessage();
if (StringUtils.containsIgnoreCase(errorMessage, Constants.FOREIGN_KEY_CONSTRAINT)) {
// A reference constaint error (unknown reporter sent data)
// 1) insert a new record into hierarchy table
HashMap map = (HashMap)data;
Integer senderId = (Integer)map.get(BaseFieldSetMapper.SENDER_ID);
if (senderId != null) {
Long reporterId = (Long)map.get(BaseFieldSetMapper.REPORTER_ID);
// via Hibernate
Field field = reporterService.getFieldBySenderId(senderId);
final Reporter reporter = new Reporter();
reporter.setField(field);
reporter.setSenderId(senderId);
reporter.setReporterId(reporterId);
reporter.setReporterType(reporterType);
// must be here
//reporterService.saveReporter(reporter);
// this is done in a new transaction
Executors.newSingleThreadExecutor().submit(new Callable<Object>() {
public Object call() throws Exception {
//via Hibernate
reporterService.saveReporter(reporter);
return null;
}
});
}
}
}
}
Hope, that there is a more elegant solution for the problem.
About the same problem I wrote in the different thread http://forum.springframework.org/sho...843#post173843 and that's why I asked if it is possible to run retry after recover, as updating the hierarchy table in a recoverer doesn't impose the table lock, the transactions a re totally isolated, and not one inside the other.
I'm looking forward to seeing your answer or tip. As the previous time with 1.0.0.m3 your answer helped me.
Thank you in advance.
Alex.
Junior Member
Transaction isolation
About your hint
Posting Permissions
Reply With Quote