Trying to configure Spring Batch Job Repository

[vybe3142]

I'm trying to setup a Spring Batch project from scratch to understand how the plumbing fits together (as opposed to using an eclipse / Spring Batch template project).

I've followed this series of tutorials upto this point http://numberformat.wordpress.com/20...ob-repository/ .

Where I'm getting stuck is trying to configure the internal job repository to work with my database (Postgres in my case). I've done this in the past by tweaking working spring batch template projects and I feel the approach presented in the tutorial is somewhat cumbersome (specifying DDLs , initializer classes etc ..., as opposed to using inbuilt Spring functionality).

However, when I attempt to execute a simple batch program, I get the following error. Looking at the DB, the tables and sequences are created (verified by wiping them and re-executing the program) but it appears that there is a duplicate attempt to create them (that fails).

Thanks in advance for any help. The XML config should be visible when the moderator approves their posting

[vybe3142]

Error Message

vinay@proton ~/workspaces/defaultworkspace/springBatchHelloWorld $ mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'exec'.
[INFO] ------------------------------------------------------------------------
[INFO] Building springBatchHelloWorld
[INFO]    task-segment: [clean, compile, exec:java]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting file set: /home/vinay/workspaces/defaultworkspace/springBatchHelloWorld/target (included: [**], excluded: [])
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 5 source files to /home/vinay/workspaces/defaultworkspace/springBatchHelloWorld/target/classes
[INFO] Preparing exec:java
[INFO] No goals needed for project - skipping
[INFO] [exec:java {execution: default-cli}]
Nov 9, 2011 9:07:31 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@acaf083: display name [org.springframework.context.support.ClassPathXmlApplicationContext@acaf083]; startup date [Wed Nov 09 09:07:31 CST 2011]; root of context hierarchy
Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [simpleJob.xml]
Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [applicationContext.xml]
Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition
INFO: Overriding bean definition for bean 'helloWorldJob': replacing [Generic bean: class [org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Generic bean: class [org.springframework.batch.core.configuration.xml.JobParserJobFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null]
Nov 9, 2011 9:07:31 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@acaf083]: org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684
Nov 9, 2011 9:07:31 AM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from class path resource [batch.properties]
Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684: defining beans [jobRepository,jobLauncher,transactionManager,dataSource,org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,org.springframework.jdbc.datasource.init.DataSourceInitializer#0,defaultTokenizer,employeeFieldSetMapper,employeeLineMapper,empReader,empProcessor,empWriter,empHeaderFooterWriter,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,step1,step2,helloWorldJob,helloWorldTasklet]; root of factory hierarchy
Nov 9, 2011 9:07:32 AM org.springframework.batch.core.launch.support.SimpleJobLauncher afterPropertiesSet
INFO: No TaskExecutor has been set, defaulting to synchronous executor.
Nov 9, 2011 9:07:32 AM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
INFO: Executing SQL script from class path resource [org/springframework/batch/core/schema-postgresql.sql]
Nov 9, 2011 9:07:32 AM org.springframework.beans.factory.support.DefaultSingletonBeanRegistry destroySingletons
INFO: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684: defining beans [jobRepository,jobLauncher,transactionManager,dataSource,org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,org.springframework.jdbc.datasource.init.DataSourceInitializer#0,defaultTokenizer,employeeFieldSetMapper,employeeLineMapper,empReader,empProcessor,empWriter,empHeaderFooterWriter,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,step1,step2,helloWorldJob,helloWorldTasklet]; root of factory hierarchy
Nov 9, 2011 9:07:32 AM org.springframework.batch.core.launch.support.CommandLineJobRunner start
SEVERE: Job Terminated in error: Error creating bean with name 'org.springframework.jdbc.datasource.init.DataSourceInitializer#0': Invocation of init method failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.jdbc.datasource.init.DataSourceInitializer#0': Invocation of init method failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1420)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
	at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:282)
	at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:574)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
	at org.springframework.jdbc.datasource.init.DataSourceInitializer.afterPropertiesSet(DataSourceInitializer.java:92)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1477)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1417)
	... 19 more
.......

[vybe3142]

My relevant pom.xml and spring config files follow

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.test</groupId>
	<artifactId>springBatchHelloWorld</artifactId>
	<version>1.0-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>springBatchHelloWorld</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<spring.framework.jdbc.version>3.0.6.RELEASE</spring.framework.jdbc.version>
		<spring.batch.version>2.1.7.RELEASE</spring.batch.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>3.8.1</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jdbc</artifactId>
			<version>${spring.framework.jdbc.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-aop</artifactId>
			<version>${spring.framework.jdbc.version}</version>
		</dependency>

		<dependency>
			<groupId>commons-dbcp</groupId>
			<artifactId>commons-dbcp</artifactId>
			<version>1.2.2</version>
		</dependency>

		<dependency>
			<groupId>postgresql</groupId>
			<artifactId>postgresql</artifactId>
			<version>9.0-801.jdbc4</version>
		</dependency>

		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${spring.batch.version}</version>
		</dependency>
		<dependency>
			<groupId>commons-lang</groupId>
			<artifactId>commons-lang</artifactId>
			<version>2.1</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring</artifactId>
			<version>2.5.6</version>
		</dependency>


	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>2.0.2</version>
				<configuration>
					<source>1.5</source>
					<target>1.5</target>
				</configuration>
			</plugin>
		</plugins>
	</build>

</project>

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
	xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:context="http://www.springframework.org/schema/context"
	xmlns:jdbc="http://www.springframework.org/schema/jdbc"
	xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/batch
http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd
">

<!-- 	<beans:bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<beans:property name="transactionManager" ref="transactionManager" />
	</beans:bean> -->

<beans:bean id="jobRepository"
    class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
    <beans:property name="databaseType" value="POSTGRES"/>
    <beans:property name="dataSource" ref="dataSource"/>
    <beans:property name="transactionManager" ref="transactionManager" />
</beans:bean>





	<beans:bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<beans:property name="jobRepository" ref="jobRepository" />
	</beans:bean>

	<beans:bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />


	<!-- -->
<!-- 	<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
		destroy-method="close">
		<beans:property name="driverClassName" value="org.postgresql.Driver" />
		<beans:property name="url" value="jdbc:postgresql://localhost/test" />
		<beans:property name="username" value="postgres" />
		<beans:property name="password" value="postgressux" />
	</beans:bean>
 -->
	<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
		destroy-method="close">
		<beans:property name="driverClassName" value="${batch.jdbc.driver}" />
		<beans:property name="url" value="${batch.jdbc.url}" />
		<beans:property name="username" value="${batch.jdbc.user}" />
		<beans:property name="password" value="${batch.jdbc.password}" />
	</beans:bean>
	
	<context:property-placeholder location="classpath:batch.properties" />

	<jdbc:initialize-database data-source="dataSource">
		<jdbc:script location="${batch.schema.script}" />
	</jdbc:initialize-database>


	<!-- -->
</beans:beans>

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/batch"
	xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="

http://www.springframework.org/schema/batch

http://www.springframework.org/schema/batch/spring-batch-2.1.xsd

http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

	<beans:import resource="applicationContext.xml" />

	<!-- Tokenizer - Converts a delimited string into a Set of Fields -->
	<beans:bean name="defaultTokenizer"
		class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" />

	<!-- FieldSetMapper - Populates a bean's attributes with using the FieldSet -->
	<beans:bean name="employeeFieldSetMapper" class="com.test.EmployeeFieldSetMapper" />

	<!-- LineMapper - Uses the tokenizer and Mapper to create instances of a 
		Bean. -->
	<beans:bean name="employeeLineMapper"
		class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
		<beans:property name="lineTokenizer" ref="defaultTokenizer" />
		<beans:property name="fieldSetMapper" ref="employeeFieldSetMapper" />
	</beans:bean>

	<!-- Reader - used by the tasklet to process one Item from the input. -->
	<beans:bean name="empReader"
		class="org.springframework.batch.item.file.FlatFileItemReader">
		<beans:property name="lineMapper" ref="employeeLineMapper" />

		<!-- use spring integrations for the following, but for now filename is 
			hard coded -->
		<beans:property name="resource" value="input_data.txt" />
	</beans:bean>

	<!-- Processor -->

	<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
	</beans:bean>

	<!-- Writer -->
	<beans:bean id="empWriter"
		class="org.springframework.batch.item.file.FlatFileItemWriter">
		<beans:property name="resource" value="file:target/output_data.txt" />
		<beans:property name="lineAggregator">
			<beans:bean
				class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
				<beans:property name="delimiter" value="|" />
				<beans:property name="fieldExtractor">
					<beans:bean
						class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
						<beans:property name="names"
							value="empId,lastName,title,salary,rank" />
					</beans:bean>
				</beans:property>
			</beans:bean>
		</beans:property>
		<beans:property name="footerCallback" ref="empHeaderFooterWriter" />
		<beans:property name="headerCallback" ref="empHeaderFooterWriter" />
	</beans:bean>

	<beans:bean id="empHeaderFooterWriter" class="com.test.EmployeeItemWriter">
		<beans:property name="delegate" ref="empWriter" />
	</beans:bean>


	<job id="helloWorldJob">
		<step id="step1" next="step2">
			<tasklet ref="helloWorldTasklet" />
		</step>
		<step id="step2">
			<tasklet>
<!-- 				<chunk reader="empReader" processor="empProcessor" writer="empWriter"
					commit-interval="1" /> -->
					
					<!-- Use the modified writer -->
            <chunk reader="empReader" processor="empProcessor" writer="empHeaderFooterWriter" commit-interval="1"/>					
			</tasklet>
		</step>
	</job>

	<beans:bean name="helloWorldTasklet" class="com.test.HelloWorldTasklet" />

	<!-- To run the job from the command line type the following: mvn clean 
		compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner 
		-Dexec.args="simpleJob.xml helloWorldJob" -->
</beans:beans>

batch.jdbc.url=jdbc:postgresql://localhost/my_springbatch_db
batch.jdbc.driver=org.postgresql.Driver
batch.jdbc.user=postgres
batch.jdbc.password=****************
batch.jdbc.testWhileIdle=false
batch.jdbc.validationQuery=
batch.drop.script=/org/springframework/batch/core/schema-drop-postgresql.sql
batch.schema.script=/org/springframework/batch/core/schema-postgresql.sql
batch.business.schema.script=business-schema-postgresql.sql
batch.data.source.init=true
batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer
batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler
batch.database.incrementer.parent=sequenceIncrementerParent
batch.grid.size=2
batch.verify.cursor.position=true

[rishishehrawat]

Is the SQL script that Spring is trying to execute working fine ? Have you verified by executing script from SQL client ?