How to process many files in one go?

[barbarella]

Hi everyone,

I am new to the Spring Batch and would be very grateful if somebody helped me.

We have a scenario when a scheduler calls periodically (once a day) a batch process to process one or more
flat files (uploaded before) in one go one by one. Each file has a unique name.

I played with some of the Spring Batch distribution samples and everything worked fine but
they all have a file name hard coded in the application context as an input source.

Could somebody give me a hint what I should do?

Thanks

[jonnio]

Can you use a property file or custom property editor to set a dynamic value?

[robert.kasanicky]

Joining the files into a single file before starting the job (or as a first step in the job) may be an option.

[barbarella]

I]Can you use a property file or custom property editor to set a dynamic value?[/I]

It seems I cannot :confused:.

As I understand property files and custom property editors are used for property externalizing in a file and value type conversions respectively and it happens once only when we load the application context.

But we have different files (names too) at each batch call by scheduler and it happens many times.

Joining the files into a single file before starting the job (or as a first step in the job) may be an option.

It sounds good.

You mean to join them together and assign the same name (hard coded in the context) to this group at each call by scheduler?

[robert.kasanicky]

You mean to join them together and assign the same name (hard coded in the context) to this group at each call by scheduler?

The name of the file does not necessarily need to be hardcoded, you can pass it to the job as parameter and use a property placeholder as input file name value in configuration.

[Dave Syer]

N.B. to parameterise input file names we provide the BatchResourcFactoryBean. By default it will look for a file with name generated from the JobIdentifier in the form /%BATCH_ROOT%/data/%JOB_NAME%/%JOB_IDENTIFIER%-%STEP_NAME%.txt. You can change the pattern and also strategise the way the JobIdentifier is turned into a label for the pattern. This fits most common scenarios of the type you describe without having to change file names, as long as you can process each file in its own job.e