Step Execution Issue with m4

[Choucri FAHED]

Hello,

I recently migrated my project from m3 to m4. And unfortunately, I cannot run any job. I constantly get this error during step execution:
13:45:20,750 INFO SimpleJobLauncher:90 - Job: [SimpleJob: [name=reencryptSiteKeyJob]] failed with the following parameters: [{}{}{}]
org.springframework.dao.OptimisticLockingFailureEx ception: Attempt to update step execution id=0 with out of date version (1)
at org.springframework.batch.execution.repository.dao .JdbcStepDao.update(JdbcStepDao.java:566)
at org.springframework.batch.execution.repository.Sim pleJobRepository.saveOrUpdate(SimpleJobRepository. java:262)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:301)
at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:182)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :149)
at org.springframework.transaction.interceptor.Transa ctionInterceptor.invoke(TransactionInterceptor.jav a:106)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
at org.springframework.aop.interceptor.ExposeInvocati onInterceptor.invoke(ExposeInvocationInterceptor.j ava:89)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy0.saveOrUpdate(Unknown Source)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.execute(SimpleStepExecutor.java:2 84)
at org.springframework.batch.execution.step.simple.Ab stractStep.execute(AbstractStep.java:131)
at org.springframework.batch.execution.job.simple.Sim pleJob.execute(SimpleJob.java:82)
at org.springframework.batch.execution.launch.SimpleJ obLauncher$1.run(SimpleJobLauncher.java:85)
at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
at org.springframework.batch.execution.launch.SimpleJ obLauncher.run(SimpleJobLauncher.java:80)
at com.amadeus.jcp.util.batch.AbstractBatchLauncherTe sts.testLaunchJob(AbstractBatchLauncherTests.java: 68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154 )
at junit.framework.TestCase.runBare(TestCase.java:127 )
at org.springframework.test.ConditionalTestCase.runBa re(ConditionalTestCase.java:76)
at junit.framework.TestResult$1.protect(TestResult.ja va:106)
at junit.framework.TestResult.runProtected(TestResult .java:124)
at junit.framework.TestResult.run(TestResult.java:109 )
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:2 08)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.eclipse.jdt.internal.junit.runner.junit3.JUnit 3TestReference.run(JUnit3TestReference.java:130)
at org.eclipse.jdt.internal.junit.runner.TestExecutio n.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.run(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.main(RemoteTestRunner.java:196)

I followed the migration guide and also applied exactly the same configuration as the one found in Spring Batch samples. Besides, I do not wish to fallback to m3 since with m4 my code and configuration got simplified! Do you have any idea where could it come from?

Cheers,
Choucri

[Dave Syer]

Can you post the configuration? I assume you must be using some asynchronous processing somewhere? It would be good to hear more details of what the step is doing.

(Please use [code][/code] tags to post code and stack traces.)

[lucasward]

Here's the point in code that throws the exception:

		synchronized (stepExecution) {

			Integer version = new Integer(stepExecution.getVersion().intValue() + 1);
			Object[] parameters = new Object[] { stepExecution.getStartTime(), stepExecution.getEndTime(),
					stepExecution.getStatus().toString(), stepExecution.getCommitCount(), stepExecution.getTaskCount(),
					PropertiesConverter.propertiesToString(stepExecution.getExecutionAttributes().getProperties()),
					stepExecution.getExitStatus().isContinuable() ? "Y" : "N",
					stepExecution.getExitStatus().getExitCode(), exitDescription, version, stepExecution.getId(),
					stepExecution.getVersion() };
			int count = jdbcTemplate.update(getUpdateStepExecutionQuery(), parameters, new int[] { Types.TIMESTAMP,
					Types.TIMESTAMP, Types.VARCHAR, Types.INTEGER, Types.INTEGER, Types.VARCHAR, Types.CHAR,
					Types.VARCHAR, Types.VARCHAR, Types.INTEGER, Types.INTEGER, Types.INTEGER });

			// Avoid concurrent modifications...
			if (count == 0) {
				throw new OptimisticLockingFailureException("Attempt to update step execution id="
						+ stepExecution.getId() + " with out of date version (" + stepExecution.getVersion() + ")");
			}
			
			stepExecution.incrementVersion();
			
		}

The query that is run will update a step execution where version equals what was passed in. The only way where count can equal 0 is if no rows were updated.

The only two things I can think of that would cause that issue are someone else updating the stepexecution (thus incrementing the version) or the StepExecution not existing. I don't believe any of this code was changed between m3 and m4 though.

Looking at the exception I see this: "Attempt to update step execution id=0 with out of date version (1)"

The JdbcStepDao will *always* insert the step execution for the first time with a version of 1. So, assuming this is the first update of the StepExecution, you can probably rule out the version being the problem. Therefore, I would guess that for some reason the execution was either never inserted (possible transaction issue) or someone tampered with the id (0 seems a bit suspect for the id.)

[Choucri FAHED]

Hello,

I use exactly the same configuration as the one in the Spring Batch samples project, except that I use 2 data sources instead of one (an in memory framework database and a business one). Here's the container-context.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:aop="http://www.springframework.org/schema/aop"
  xmlns:tx="http://www.springframework.org/schema/tx"
  xmlns:p="http://www.springframework.org/schema/p"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
		http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
		http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">

  <import resource="data-source-context.xml" />

  <bean id="stepScope"
    class="org.springframework.batch.execution.scope.StepScope" />

  <bean id="jobConfigurationRegistryBeanPostProcessor" abstract="true"
    class="org.springframework.batch.execution.configuration.JobRegistryBeanPostProcessor">
    <property name="jobConfigurationRegistry"
      ref="jobConfigurationRegistry" />
  </bean>

  <bean id="jobLauncher"
    class="org.springframework.batch.execution.launch.SimpleJobLauncher">
    <property name="jobRepository" ref="jobRepository" />
  </bean>

  <bean id="jobConfigurationRegistry"
    class="org.springframework.batch.execution.configuration.MapJobRegistry" />

  <aop:config>
    <aop:advisor
      pointcut="execution(* org.springframework.batch.execution..*Repository+.*(..))"
      advice-ref="frameworkTransactionAdvice" />
  </aop:config>

  <tx:advice id="frameworkTransactionAdvice"
    transaction-manager="frameworkTransactionManager">
    <tx:attributes>
      <tx:method name="*Create*" propagation="REQUIRES_NEW"
        isolation="SERIALIZABLE" />
      <tx:method name="*" />
    </tx:attributes>
  </tx:advice>

  <bean id="jobRepository"
    class="org.springframework.batch.execution.repository.SimpleJobRepository">
    <constructor-arg ref="jobDao" />
    <constructor-arg ref="stepDao" />
  </bean>

  <bean id="jobDao" lazy-init="true"
    class="org.springframework.batch.execution.repository.dao.JdbcJobDao">
    <property name="jdbcTemplate" ref="frameworkJdbcTemplate" />
    <property name="jobIncrementer" ref="jobIncrementer" />
    <property name="jobExecutionIncrementer"
      ref="jobExecutionIncrementer" />
  </bean>

  <bean id="stepDao" lazy-init="true"
    class="org.springframework.batch.execution.repository.dao.JdbcStepDao">
    <property name="jdbcTemplate" ref="frameworkJdbcTemplate" />
    <property name="stepIncrementer" ref="stepIncrementer" />
    <property name="stepExecutionIncrementer"
      ref="stepExecutionIncrementer" />
    <property name="jobDao" ref="jobDao" />
  </bean>

  <bean id="frameworkJdbcTemplate"
    class="org.springframework.jdbc.core.JdbcTemplate">
    <property name="dataSource" ref="frameworkDataSource" />
  </bean>

  <bean id="simpleJob"
    class="org.springframework.batch.execution.job.simple.SimpleJob"
    abstract="true">
    <property name="jobRepository" ref="jobRepository" />
    <property name="restartable" value="true" />
  </bean>

  <bean id="abstractStep" abstract="true">
    <property name="jobRepository" ref="jobRepository" />
    <property name="transactionManager"
      ref="frameworkTransactionManager" />
    <property name="allowStartIfComplete" value="true" />
    <property name="saveExecutionAttributes" value="true" />
  </bean>

  <bean id="simpleStep" parent="abstractStep"
    class="org.springframework.batch.execution.step.simple.SimpleStep"
    abstract="true">
    <property name="exceptionHandler">
      <bean
        class="org.springframework.batch.repeat.exception.handler.SimpleLimitExceptionHandler">
        <property name="limit" value="5" />
        <property name="useParent" value="true" />
      </bean>
    </property>
    <property name="commitInterval" value="1" />
  </bean>

  <bean id="repeatOperationsStep" parent="abstractStep"
    class="org.springframework.batch.execution.step.simple.RepeatOperationsStep"
    abstract="true">
  </bean>

  <bean id="customEditorConfigurer"
    class="org.springframework.beans.factory.config.CustomEditorConfigurer">
    <property name="customEditors">
      <map>
        <entry key="int[]">
          <bean
            class="org.springframework.batch.support.IntArrayPropertyEditor" />
        </entry>
        <entry
          key="org.springframework.batch.io.file.transform.Range[]">
          <bean
            class="org.springframework.batch.io.file.transform.RangeArrayPropertyEditor" />
        </entry>
        <entry key="java.util.Date">
          <bean
            class="org.springframework.beans.propertyeditors.CustomDateEditor">
            <constructor-arg>
              <bean class="java.text.SimpleDateFormat">
                <constructor-arg value="yyyyMMdd" />
              </bean>
            </constructor-arg>
            <constructor-arg value="false" />
          </bean>
        </entry>
      </map>
    </property>
  </bean>

</beans>

And here's the data-source-context.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:util="http://www.springframework.org/schema/util"
  xmlns:context="http://www.springframework.org/schema/context"
  xsi:schemaLocation="http://www.springframework.org/schema/context http://www.springframework.org/schema/beans/spring-context-2.5.xsd
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
	http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd">

  <bean id="businessDataSource"
    class="org.springframework.jdbc.datasource.DriverManagerDataSource"
    autowire-candidate="false">
    <property name="driverClassName" value="${batch.jdbc.driver}" />
    <property name="url" value="${batch.jdbc.url}" />
    <property name="username" value="${batch.jdbc.user}" />
    <property name="password" value="${batch.jdbc.password}" />
  </bean>

  <bean id="businessTransactionManager"
    class="org.springframework.jdbc.datasource.DataSourceTransactionManager"
    lazy-init="true" autowire-candidate="false">
    <property name="dataSource" ref="businessDataSource" />
  </bean>

  <bean id="frameworkDataSource"
    class="org.springframework.jdbc.datasource.DriverManagerDataSource">
    <property name="driverClassName" value="${framework.jdbc.driver}" />
    <property name="url" value="${framework.jdbc.url}" />
    <property name="username" value="${framework.jdbc.user}" />
    <property name="password" value="${framework.jdbc.password}" />
  </bean>

  <bean id="sessionFactory"
    class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
    <property name="dataSource" ref="frameworkDataSource" />
    <property name="mappingLocations" value="classpath*:/*.hbm.xml" />
    <property name="hibernateProperties">
      <value>
        <![CDATA[
      hibernate.show_sql=true
      hibernate.format_sql=true
      ]]>
      </value>
    </property>
  </bean>

  <bean id="frameworkTransactionManager"
    class="org.springframework.orm.hibernate3.HibernateTransactionManager"
    lazy-init="true">
    <property name="sessionFactory" ref="sessionFactory" />
  </bean>

  <bean id="batchProperties"
   class="org.springframework.beans.factory.config.PropertyOverrideConfigurer">
    <property name="location" value="classpath:batch.properties" />
    <property name="localOverride" value="true" />
    <property name="properties">
      <bean class="java.lang.System" factory-method="getProperties" />
    </property>
    <property name="ignoreInvalidKeys" value="true" />
    <property name="order" value="2" />
  </bean>

  <bean id="overrideProperties"
    class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="location" value="classpath:batch.properties" />
    <property name="systemPropertiesModeName"
      value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
    <property name="ignoreUnresolvablePlaceholders" value="false" />
    <property name="order" value="1" />
  </bean>

  <bean id="incrementerParent"
    class="${framework.database.incrementer.class}" abstract="true">
    <property name="dataSource" ref="frameworkDataSource" />
  </bean>

  <bean id="jobIncrementer" parent="incrementerParent">
    <property name="incrementerName" value="BATCH_JOB_SEQ" />
  </bean>

  <bean id="jobExecutionIncrementer" parent="incrementerParent">
    <property name="incrementerName" value="BATCH_JOB_EXECUTION_SEQ" />
  </bean>

  <bean id="stepIncrementer" parent="incrementerParent">
    <property name="incrementerName" value="BATCH_STEP_SEQ" />
  </bean>

  <bean id="stepExecutionIncrementer" parent="incrementerParent">
    <property name="incrementerName" value="BATCH_STEP_EXECUTION_SEQ" />
  </bean>

  <bean id="partitionIncrementer" parent="incrementerParent">
    <property name="incrementerName" value="BATCH_PARTITION_SEQ" />
  </bean>

  <bean id="partitionInstanceIncrementer" parent="incrementerParent">
    <property name="incrementerName"
      value="BATCH_PARTITION_EXECUTION_SEQ" />
  </bean>

</beans>

As you can see, I have the same configuration as 'samples' especially when it comes to transaction management. Also, since I use an in memory database, how could the id be tempered with?

Thanks for your help!
Choucri

[lucasward]

Interesting, if you have two different datasources, who is committing on the business data source?

Also, on the bottom of the data source configuration I saw two 'partition incrementers', are those used somewhere?

[Choucri FAHED]

Indeed, I don't know how it works exactly. But I asked this question before on this forum:
http://forum.springframework.org/showthread.php?t=48078
and it solved my issue! Thanks to your help, I managed to make it work fine with m3.

When it comes to the 'partition incrementers' beans, you are right I don't use them anywhere. However, since I'm totally new to Spring and I was becoming a bit desperate, I decided to apply exactly the same configuration as samples. Then, do a cleanup once I solve my issue.

Cheers,
Choucri