Apr 18th, 2008, 10:39 AM
Files having different columns
There are two files File1 and File2. Some of the columns are common among these files. Job will have to process each of these files separately and load data into the database in the same table.
File1 columns are A,B and C.
File2 columns are D,B and C.
1) What would be the best approach to impement this in terms of ItemReader, Tokenizer and Domain/Value object?
2) Do i create two FlatFileItemReader (one for each file - File1 and File2 in the DelegatingItemReader), which will handle seprate tokenizer (for File1 and File2)?
Last edited by springforever; Apr 18th, 2008 at 11:14 AM.
Apr 18th, 2008, 11:08 AM
I would go with your option of creating two flatfileitemreaders. It is hard to provide any input without know what you are trying to do here.
What is the processing involves? whether you are updating database? If yes then does B and C are party of the primary key, then you might have to handle the uniqueness of these keys as they are coming from different sources.
Apr 18th, 2008, 11:16 AM
I have corrected my post above. I wpuld like to load File1 and File2 one by one using the same job.
Originally Posted by hailspring
Apr 18th, 2008, 11:24 AM
Go through TradeJob sample in the distribution file.
If you want to do it separately one after the another, then best way is to have two steps in one job
This will read file 1 with its own item reader, tokenizer and domain object
This will read file 2 with its own item reader, tokenizer and domain object.
hope that helps. TradeJob sample exactly does this.
Apr 18th, 2008, 11:30 AM
By separately i meant on day 1, job will receive File1. On day 2, job will receive File2.
Originally Posted by hailspring
Apr 18th, 2008, 01:06 PM
Shouldnt your scheduler take care of it?
It should be able to pass-in file-name as the input and you can generalize your domain object and job to process it. In this case you may not need more than one step or job.
From my understanding you can instruct your scheduler to run a particular job on specific days and depending upon the day you can pass in your fileLocator to be either File1 or File2.
Hope that helps!
Apr 18th, 2008, 01:14 PM
I agree with Hailspring, I would pass-in either the file name, or the file type as a JobParameter. This is important because then Spring Batch will treat them as separate instances (very important for restart).
The one problem I think you're struggling with is that even if you're able to be given the filename at runtime, you may have two different tokenizers that must be used. In this case, I would make a composite LineTokenizer(you never said whether it was delimited or fixed width) that is also a StepExecutionListener, you can then decide which tokenizer to use in the beforeStep by checking the JobParameters.
Apr 18th, 2008, 02:17 PM
Yes, that's what i was looking for. But, can i rather have CustomizedDelItemReader (extending DelItemReader) which decide the ItemReader to delegate to (in a way CompositeItemWriter does)? In this case, i will have different tokenizer (as in multilineOrderJob example) configured in XML.
Originally Posted by lucasward
How do i make compositeLineTokenizer? Do i create a customized class extending PreFixMatchingComLineTok...?
How do i set Tokenizer in the StepExeListener?
Last edited by springforever; Apr 18th, 2008 at 02:42 PM.
Reason: Added Question
Apr 18th, 2008, 03:30 PM
Great! It's working. Went with your approach, Lucas.