reading from multiple files of different format

[nikunj_c]

Hello All,

I have a scenario similar to the following example scenario:

Read a remote XML file with 2 columns with the first column as key.
Read a local flat file with 4 columns with one column as matcing key from above file.
Merge records from each file by matching key columns so that the pojo has now 5 properties (4 + 1 one key column common to both files).
Write the output to a database.

I am confused as to how the flow/job configuration will be and how will I be reading multiple files of different formats.


FootballJob exactly matches my requirement. But here I am talking about huge inputs (like 50000 records of 250 columns). So I am skeptic as to whether the approach taken in FootballJob will be efficient.


Please help!!

Thanks!
Nik