Spring Batch and MongoDB

[Tobias Trelle]

Hi,

I wrote a blog post explaining how to bring together MongoDB (one of these NoSQL datastores) and Spring Batch:

http://blog.codecentric.de/en/2012/1...batch-mongodb/

The ready-to-use source code can be found in this github repo: https://github.com/ttrelle/spring-batch-mongodb-support

Any feedback is welcome.

Cheers
Tobias

[mminella]

Interesting stuff. A couple small recommendations:

1. In the MongoDBItemReader.doOpen, you do some checks that I would do in the afterPropertiesSet (specifically checking that the skip is not less than 0 and checking that db exists (the dbExists one could be done in either place really).
2. I see in the MongoDBItemWriter you mimic the behavior in the TransactionAwareBufferedItemWriter with delaying the actual write to once the transaction has been completed. We just updated that functionality slightly so you may want to take a look at the latest code. The issue we ran into was that when the TransactionSycnrhonization's afterCompletion method was called, all exceptions it threw were being swallowed up. This causes a job to complete with an exit status of "COMPLETE" even though the data was not persisted. Take a look at how we handle it now by moving the actual persistence to beforeCommit (last thing done before the actual commit) to allow exceptions to cause a rollback of the chunk. This prevents us from having to register a chunk listener to handle the exception case (this was part of BATCH-1799 and pull request 71).
3. In the MongoDBItemWriter.prepareDocuments, I would expect the if/else logic within the for statement to exist in the converter. You could provide a base implementation that handles that and allows for additional converting. This would make a converter required but clean that up a bit. Just a thought.

Overall nice stuff. As an FYI, we are planning on supporting MongoDB (and other data stores) via Spring Data in the next release (2.2). You can follow the issue here: BATCH-1728.