View Full Version : Use parameters in job.xml using %%
sandrine
Mar 11th, 2008, 04:14 AM
Hi,
I use m5 version of spring batch .
I run my batch jobs with the parameters : /monjob.xml monjob batch.scheduleDate=200803111000.
In my job.xml file, I would like to use the batch.scheduleDate value in the name of an output file.
For that, I have replaced ${batch.scheduleDate} in the name of the ouputFile (in m3 version) by %batch.scheduleDate% but it doesn't work.
Here is my job.xml file.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aop="http://www.springframework.org/schema/aop"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
<import resource="classpath:simple-container-definition.xml" />
<bean id="ExtractionClientsJob" parent="simpleJob">
<property name="steps">
<list>
<bean id="Extraction" parent="simpleStep">
<property name="streams" ref="clientFileWriter"/>
<property name="commitInterval" value="2" />
<property name="allowStartIfComplete" value="false" />
<property name="itemReader" ref="DbItemReader" />
<property name="itemWriter" ref="clientFileWriter"/>
</bean>
</list>
</property>
</bean>
<!-- INFRASTRUCTURE SETUP -->
<bean id="DbItemReader" class="com.natixis.sphinx.batch.reader.DbItemReader">
<property name="dataSource" ref="dataSource" />
<property name="sql">
<value> select p.ceseqo, p.cocivi,p.nompat,p.nommar,p.prenom,p.preno2,p.dah eme,p.cosecu,p.cositf,a.librue,a.copost,a.libvil,a .copays,a.coposn,a.copayn,a.libvin,c.cochar,c.dahe m1,c.numrib from sphmclient c, sphmadresse a, sphmpersonne p where c.cochar=7 and c.copers=p.ceseqo and p.ceseqo=a.copers order by p.ceseqo</value>
</property>
<property name="mapper">
<bean class="com.natixis.sphinx.batch.modele.mapping.ClientRowM apper" />
</property>
</bean>
<bean id="clientFileWriter" class="org.springframework.batch.io.file.FlatFileItemWrit er">
<property name="resource" ref="resourceSortie1" />
<property name="lineAggregator" >
<bean class="org.springframework.batch.io.file.transform.Delimi tedLineAggregator">
<property name="delimiter" value=";"/>
</bean>
</property>
<property name="fieldSetUnmapper" >
<bean class="com.natixis.sphinx.batch.modele.mapping.ClientFiel dSetCreator" />
</property>
<property name="shouldDeleteIfExists" value="true"/>
</bean>
<bean id="mapContext" class="com.natixis.sphinx.batch.domain.Context">
<property name="map">
<map>
<entry key="resourceSortie1" value-ref="resourceSortie1"/>
<entry key="dataSource" value-ref="dataSource"/>
</map>
</property>
</bean>
<bean id="resourceSortie1" class="java.lang.String">
<constructor-arg value="file:data/output/%batch.scheduleDate%.clients.txt"/>
</bean>
</beans>
I have read that the class org.springframework.batch.execution.resource.StepE xecutionProxyResource replace %% by the value of parameters but I don't know what I have to do to be able to use it.
Could you give me a sample which shows how to use parameters of batch in job.xml?
Thanks in advance (sorry for my bad english).
jonnio
Mar 11th, 2008, 05:28 AM
I think you mean
%{batch.scheduleDate}
sandrine
Mar 11th, 2008, 06:28 AM
I have tried:
%{batch.scheduleDate}
and
%{batch.scheduleDate}%
but it isn't replaced by the value.
lucasward
Mar 11th, 2008, 05:41 PM
You need to use the actual StepExecutionResourceProxy, you're just creating a String. It should look something like:
<bean id="resourceSortie1" class="org.springframework.batch.core.support.StepExecuti onResourceProxy">
<property name="filePattern" value="data/output/%batch.scheduleDate%.clients.txt" />
</bean>
I haven't checked that to ensure it's valid, but I think you get the idea. The spring container won't automatically convert your string using the proxy, you have to declare it directly.
lucasward
Mar 11th, 2008, 05:44 PM
I almost forget, you need to register the proxy with the step as well, so that it can get access to the job parameters.
sandrine
Mar 12th, 2008, 02:51 AM
Thanks for your answers.
I tried the following job.xml:
<bean id="ExtractionClientsJob" parent="simpleJob">
<property name="steps">
<list>
<bean id="Extraction" parent="simpleStep">
<property name="listeners" ref="resourceSortie1"/> <property name="commitInterval" value="2" />
<property name="allowStartIfComplete" value="false" />
<property name="itemReader" ref="DBInputTemplate" />
<property name="itemWriter" ref="clientFileWriter"/>
</bean>
</list>
</property>
</bean>
...
<bean id="resourceSortie1" class="org.springframework.batch.execution.resource.StepE xecutionProxyResource">
<property name="filePattern" value="data/output/%batch.scheduleDate%.clients.txt"/> </bean>
But the error is always:
The delegate resource has not been initialised. Remember to register this object as a StepListener.
And I don't understand how to register this object as a StepListener.
Could you clarify this point please?
Dave Syer
Mar 12th, 2008, 03:08 AM
You have registered the listener in the fragement you showed. Maybe we are missing something else? From the stack trace you can probably see where the resource is being used - my guess it is being used outside the step somewhere?
sandrine
Mar 12th, 2008, 03:19 AM
Yes it's used by clientFileWriter which is a FlatFileItemWriter:
<bean id="clientFileWriter" class="org.springframework.batch.io.file.FlatFileItemWrit er">
<property name="resource" ref="resourceSortie1"/>
<property name="lineAggregator" >
<bean class="org.springframework.batch.io.file.transform.Delimi tedLineAggregator">
<property name="delimiter" value=";"/>
</bean>
</property>
<property name="fieldSetUnmapper" >
<bean class="com.natixis.sphinx.batch.modele.mapping.ClientFiel dSetCreator" />
</property>
<property name="shouldDeleteIfExists" value="true"/>
</bean>
But a flatFileItemWriter isn't a stepListener and can't register stepListener. Then I don't know what I must do.
lucasward
Mar 12th, 2008, 09:48 AM
You need to register the proxy as a StepListener, separate from the writer.
sandrine
Mar 12th, 2008, 10:19 AM
Sorry but I don't understand the code I must write to register the proxy.
Could you give me the code I must write in my job.xml or explain what I must do exactly (perhaps code in job.xml is good and I must create a itemReader class which implements ItemReadListener and then use my itemReader class instead of FlatFileItemReader)?:confused:
lucasward
Mar 12th, 2008, 12:23 PM
I don't have time to look back at M5, but the latest snapshot has a simple setListeners method. Something like:
defaultStepFactoryBean.setListeners(StepExecutionR esourceProxy);
Of course this would be in xml and not code, but I think you get the idea. I believe in M5 you would need to set that directly on the ItemOrientedStep.
Dave Syer
Mar 12th, 2008, 04:44 PM
Yes it's used by clientFileWriter which is a FlatFileItemWriter
And is that file writer used somewhere outside a step? The stack trace might help.
sandrine
Mar 13th, 2008, 06:19 AM
here is the log:
ERROR - BatchMain.start(170) | Job Terminated in error:
org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'ExtractionClientsJob' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot create inner bean 'Extraction' of type [org.springframework.batch.execution.step.support.D efaultStepFactoryBean] while setting bean property 'steps' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'Extraction' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot resolve reference to bean 'clientFileWriter' while setting bean property 'itemWriter'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveInnerBean(BeanDefinitio nValueResolver.java:229)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:117)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveManagedList(BeanDefinit ionValueResolver.java:286)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:126)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1274)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1042)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
at java.security.AccessController.doPrivileged(Access Controller.java:197)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.preInstantiateSingletons(Defaul tListableBeanFactory.java:413)
at org.springframework.context.support.AbstractApplic ationContext.finishBeanFactoryInitialization(Abstr actApplicationContext.java:735)
at org.springframework.context.support.AbstractApplic ationContext.refresh(AbstractApplicationContext.ja va:369)
at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationContext.java:122)
at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationContext.java:66)
at com.natixis.sphinx.batch.BatchMain.start(BatchMain .java:146)
at com.natixis.sphinx.batch.BatchMain.main(BatchMain. java:212)
Caused by:
org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'Extraction' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot resolve reference to bean 'clientFileWriter' while setting bean property 'itemWriter'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:274)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:104)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1274)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1042)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
at java.security.AccessController.doPrivileged(Access Controller.java:197)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveInnerBean(BeanDefinitio nValueResolver.java:219)
... 20 more
Caused by:
org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.initializeBean(Abstract AutowireCapableBeanFactory.java:1362)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:540)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
at java.security.AccessController.doPrivileged(Access Controller.java:197)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:268)
... 28 more
Caused by:
java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
at org.springframework.util.Assert.state(Assert.java: 355)
at org.springframework.batch.execution.resource.StepE xecutionProxyResource.getFile(StepExecutionProxyRe source.java:132)
at org.springframework.batch.io.file.FlatFileItemWrit er.afterPropertiesSet(FlatFileItemWriter.java:96)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.invokeInitMethods(Abstr actAutowireCapableBeanFactory.java:1390)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.initializeBean(Abstract AutowireCapableBeanFactory.java:1359)
... 37 more
I use a DefaultStepFactoryBean which has a listeners property like ItemOrientedStep has stepListeners property.
I have seen that in StepExecutionProxyResource there is a field named "delegate" which must not be null but there isn't setDelegate method in this class and so it is always null!!!
If I am wrong, could you say how to set the delegate?
Else could you add the setDelegate method in the next version?
I think I'm going to wait for the next version of spring Batch to use parameter of batch instead of system's parameter.
Sorry I should have had to see this problem before.
robert.kasanicky
Mar 13th, 2008, 09:15 AM
Delegate is set when the step starts and calls the beforeStep(..) callback, it's ok there is no setter for it.
fgarsombke
Mar 13th, 2008, 11:11 AM
This works just fine.
Here is the job:
<bean id="fixedLengthFaresAssessorToBlsJob" parent="simpleJob">
<property name="steps">
<bean id="stepReadFaresAssessorAndWriteToBls" parent="simpleStep">
<property name="listeners">
<list>
<ref bean="fileResource"/>
</list>
</property>
<!-- need a stream to open record before we read it in -->
<property name="streams" ref="faresAssessorFlatFileItemReader"/>
<property name="itemReader" ref="faresAssessorToBlsItemReader"/>
<property name="itemWriter" ref="blsWriter"/>
</bean>
</property>
</bean>
and resource:
<bean id="fileResource" class="org.springframework.batch.execution.resource.StepE xecutionProxyResource">
<property name="filePattern" value="data/%file.name%"/>
</bean>
<bean id="faresAssessorFlatFileItemReader" class="org.springframework.batch.io.file.FlatFileItemRead er">
<property name="resource" ref="fileResource" />
<property name="lineTokenizer" ref="faresAssessorFixedFileDescriptor" />
<property name="fieldSetMapper">
<bean class="org.springframework.batch.io.file.mapping.PassThro ughFieldSetMapper" />
</property>
</bean>
Then just add job parameters as such:
jpb.addString("file.name", "fixedLengthFaresAssessorToBlsSmallData.txt");
launcher.run(job,jpb.toJobParameters());
Thanks.
Franz Garsombke
sandrine
Mar 14th, 2008, 03:45 AM
I have tried with a reader and it works but it doesn't work with a writer.
Why?
liang
Apr 10th, 2008, 09:10 PM
Because there is a small bug before spring-batch 1.0.1. I have created an issue in the jira. Here is the link http://jira.springframework.org/browse/BATCH-560
This problem will be fixed in 1.0.1
sandrine
Apr 11th, 2008, 12:37 AM
Thank you.
Powered by vBulletin® Version 4.2.1 Copyright © 2013 vBulletin Solutions, Inc. All rights reserved.