Jul 16th, 2011, 09:29 AM
Big project evaluation questions, expecting to use spring batch.
Hi, i work for a big telecom company and i've been evaluating tools that would help me do a project migration of over 200+ jobs that do the following:
- Runs at an interval/time schedule.
- Reads data from different input sources(flat file in directory, db, ftp).
- Processes data in different ways(mainly formatting such as date format, string length, trimming, sometimes business logic applies but most of the time the business logic required is in the query conditions or sometimes the files just need some formating).
- Output data in different places(FTP, inserts in the database, write to another file in a directory, Email, Web Service).
Spring batch contains a really good interface for us to standarize our input and output to the different components that we create. and probably we could reuse those components(since we do not pretend to create domain objects that represent the tables we read, we might just use readers that return Map<String, Object> values, or a more efficient domain object like a KeyValue object).
And well i really think that this is the framework that we should stay with, there is a possiblity of custom made code that might satisfy our needs but i personally rather use a framework that is supported by people and has a community, much easier to develop this way since we can help out on the open source community with bug reports and probably donate code to the project instead of taking it all in for ourselves.
I was hoping that this book -> http://www.amazon.com/Pro-Spring-Bat...0825619&sr=8-1 would
be out by the date they specified but aparently i'll have to wait to speed up with spring batch learning.
Note: Once we finish the project and have it in production with most bug fixes we pretend to release it as a sample spring batch project for further research and improvement.
So to start out i would like to receive some suggestions if it is possible:
- Do you have production code that uses spring batch in your company?
- How reusable was it? Which workarounds did you have to do in order to make it as reusable as posible?
- With late binding i pretend to reuse a job as much as possible(Example: FlatFileToDB Jobs will read different flat files and send to different db configurations with configured insert PreparedStatements in paralell), have you done this in production code, and how has this worked for you?
- If i use late binding with the same job in a multithreaded application, is this thread safe? or should i specify a prototype scope to certain objects.
- We've been looking at a way to integrate spring-batch with something like OSGi to prevent downtime when there is a job modification, have you done this? and how. If you've found guidance on the internet please guide us to the link.
This are the first batch of questions that we have for the community, we do not expect you to answer them all, but any help to resolve this will be greatly appreciated.