Nov 14th, 2012, 03:01 AM
How would you implement this scenario by using spring-batch?
I'm using this environment:
- spring 3.1.1
- myBatis 3.1.1
- spring-myBatis 1.1.1
- spring-batch 2.1.9
I state that I'm newbie in spring-batch so I'ld like to have a comparision with you experts about how you'ld implement my own scenario by using spring-batch
I have a scheduled job to do every week; my job steps are:
- in the first step is execute a query in order to retrieve user ids. This API is provided by another partner in the project and virtually I may have all the customer registered in my application (also 2 million of ids..I know it's incredible but...they don't want to change the API by adding some filters so I can do nothing)
- in the second step I need to cicle the recovered List<Long> if ids and id by id I need to do some query on a DB I don't know by using some API provided by another partner in order to create some matrices (by using commons math);
- in the third step the created matrices need to be saved on a graph db
- in the fourth step i need to query on the created graph and then to do some calculation
Note that involved data can be very huge (I may have to handle millions of records and I have no chance to change provided queries except the ones I use after I store data in the graph DB)
In the showed scenario (I hope I was clear enough)....how would you implement it by using spring-batch?
Thank you for the support