Eliminating duplicate records before writing to DB

[abnig]

Any suggestions on how to eliminate duplicate records before passing it to the writer.

I tried implementing ItemWriter but its called as many times as the number of source records if injected into the chunk as writer.

Thanks
AN

[nikunj_c]

I believe the only way to do is to dump the whole file into a temp table in step one. And then do a self join and a write to final database in second step.

step 1: File to DB
step 2: DB to DB

Nik