A DB Reader and a file reader in the same step?

[benmasi]

Hi everybody,

I would like to implement a step which :

- reads a list of String from a file ( 'xxx', 'yyy', 'zzz', ...)
- for each of these numbers, executes a database SQL request like : select * from P where myField = 'xxx')
- then processes the results
- writes ouput to a file

I do not know how implementing the reading phase. Is it possible to have two readers (here a file reader, and a database reader) in the same step? If yes, how to implement it? Is there an example? In fact, i saw that multiple writers is possible, or multiple file readers thx to MultiResourceItemWriter, but nothing for two kind of readers inthe same step...

What is the correct way in spring batch to implement that?

Thanks for your answer.

[Marten Deinum]

IMHO it is easier to create a single reader, multiple processors (the first selecting the item from database, passing it to the next) then write.

[benmasi]

Hi Marten, thx for your reply. But, in the absolute, is it possible to have these 2 kind of readers in the same step? If yes, how? Thx to a custom reader? Spring batch does not seem to provide a class which implements that..

Regarding your suggestion, is it correct (from a architectural view) to have a reader (here the DB reader) in a processor?

[bwawok]

Many options.

1) Make your own reader. It is composed of a database reader and a file reader. Each time read() is called, the file reader reads 1 line, does a DB query, and passes both results as the return to the method.
2) Just use the file reader. In your processor, make a call to a DAO, to get whatever you need from the database.

In both of these cases you will be doing 1 DB call per row, which is a case I personally try to avoid.. depends on your volume and performance requirements.

[Marten Deinum]

Forget the fact that it is a reader, it isn't... It is more or less a processor/transformer which transforms the id (or whatever it is) into an object...

You could write a reader which chains readers but you cannot have one reader give input to another reader (at least not without much hacking). Imho the way I suggest is the easiest. (Or create a reader which does both as suggest by bwawok, you might be able to read the file, construct a single query and create a paging reader or cursos based reader).

[benmasi]

Thank you both. I am going to try implementing a 'super' reader as you suggest. Bawkok, you re right reagrding performance, i am going to use instead only one SQL request including the list of all the strings.