Files having different columns

[springforever]

There are two files File1 and File2. Some of the columns are common among these files. Job will have to process each of these files separately and load data into the database in the same table.

File1 columns are A,B and C.
File2 columns are D,B and C.

Questions:
1) What would be the best approach to impement this in terms of ItemReader, Tokenizer and Domain/Value object?
2) Do i create two FlatFileItemReader (one for each file - File1 and File2 in the DelegatingItemReader), which will handle seprate tokenizer (for File1 and File2)?

[hailspring]

I would go with your option of creating two flatfileitemreaders. It is hard to provide any input without know what you are trying to do here.

What is the processing involves? whether you are updating database? If yes then does B and C are party of the primary key, then you might have to handle the uniqueness of these keys as they are coming from different sources.

Good luck!

[springforever]

I would go with your option of creating two flatfileitemreaders. It is hard to provide any input without know what you are trying to do here.

What is the processing involves? whether you are updating database? If yes then does B and C are party of the primary key, then you might have to handle the uniqueness of these keys as they are coming from different sources.

Good luck!

I have corrected my post above. I wpuld like to load File1 and File2 one by one using the same job.

[hailspring]

Go through TradeJob sample in the distribution file.

If you want to do it separately one after the another, then best way is to have two steps in one job

Step1:
This will read file 1 with its own item reader, tokenizer and domain object

Step 2:
This will read file 2 with its own item reader, tokenizer and domain object.

hope that helps. TradeJob sample exactly does this.

Good luck!

[springforever]

Go through TradeJob sample in the distribution file.

If you want to do it separately one after the another, then best way is to have two steps in one job
Good luck!

By separately i meant on day 1, job will receive File1. On day 2, job will receive File2.

[hailspring]

Shouldnt your scheduler take care of it?

It should be able to pass-in file-name as the input and you can generalize your domain object and job to process it. In this case you may not need more than one step or job.

From my understanding you can instruct your scheduler to run a particular job on specific days and depending upon the day you can pass in your fileLocator to be either File1 or File2.

Hope that helps!

[lucasward]

I agree with Hailspring, I would pass-in either the file name, or the file type as a JobParameter. This is important because then Spring Batch will treat them as separate instances (very important for restart).

The one problem I think you're struggling with is that even if you're able to be given the filename at runtime, you may have two different tokenizers that must be used. In this case, I would make a composite LineTokenizer(you never said whether it was delimited or fixed width) that is also a StepExecutionListener, you can then decide which tokenizer to use in the beforeStep by checking the JobParameters.

[springforever]

The one problem I think you're struggling with is that even if you're able to be given the filename at runtime, you may have two different tokenizers that must be used. In this case, I would make a composite LineTokenizer(you never said whether it was delimited or fixed width) that is also a StepExecutionListener, you can then decide which tokenizer to use in the beforeStep by checking the JobParameters.

Yes, that's what i was looking for. But, can i rather have CustomizedDelItemReader (extending DelItemReader) which decide the ItemReader to delegate to (in a way CompositeItemWriter does)? In this case, i will have different tokenizer (as in multilineOrderJob example) configured in XML.

How do i make compositeLineTokenizer? Do i create a customized class extending PreFixMatchingComLineTok...?
How do i set Tokenizer in the StepExeListener?

[springforever]

Great! It's working. Went with your approach, Lucas.
Thanks hailspring!