Hi, I've got to create a batch script that fetches data from a remote web service. One of the parameters of the webservice is minTimestamp, which tells the webservice to return only the rows which have been created/updated after minTimestamp.
The other parameter is batchSize, which tells the service how many rows I want to receive.
It returns rows of data, in such a way that the timestamp of a row is contained within it:
After reading <batchSize> rows, in order to get the next batch, the timestamp from the last received row has to be passed to the webservice.PHP Code:{
"data": "...",
"timestamp" : 1234
}
This job has to be run very often as the webservice produces a lot of data.
So I was wondering what would be the "right" way to do it Spring Batch? In particular how to store state(timestamp) between job runs and how to handle retries.
I was thinking of maybe adding two steps around the main read/write step in order to save/fetch the timestamp to/from the database and placing it in the JobExecutionContext. But I was hoping maybe there is an easier builtin solution that I have overlooked.
Any ideas?


Reply With Quote