Running batch samples in Oracle 10g

[mdindagur]

Hi,

I'm trying to run the sample jobs obtained from source at http://springframework.svn.sourceforge.net/svnroot/springframework/spring-batch/trunk/. Initially I downloaded the framework along with samples; Using maven and jdk 1.5 I was able to compile, run the associated tests with HSQLDB(default). I was still getting exceptions trying to update the in-memory database. Since I need to implement my application using Oracle 10g, I thought of using oracle to see how the samples work. I modified the following to point to oracle database.

\samples\src\main\resources\batch.properties .
\execution\src\main\resources\batch.template.prope rties
\samples\src\main\resources\data-source-context-init.xml
corrected the dialect in \samples\src\main\resources\hibernate-context.xml to point to oracle
\execution\src\main\resources\schema-oracle10g.sql

I am getting the exception listed below.

org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'jobLauncher' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'simpleJobRepository' while setting bean property 'jobRepository'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'simpleJobRepository' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'sqlJobDao' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'sqlJobDao' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'jobIncrementer' while setting bean property 'jobIncrementer'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'jobIncrementer' defined in class path resource [data-source-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.InvalidPropertyException : Invalid property 'columnName' of bean class [org.springframework.jdbc.support.incrementer.Oracl eSequenceMaxValueIncrementer]: No property 'columnName' found
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:274)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:104)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1274)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1042)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.getBeansOfType(DefaultListableB eanFactory.java:296)
at org.springframework.context.support.AbstractApplic ationContext.getBeansOfType(AbstractApplicationCon text.java:950)
at org.springframework.context.support.AbstractApplic ationContext.registerListeners(AbstractApplication Context.java:711)
at org.springframework.context.support.AbstractApplic ationContext.refresh(AbstractApplicationContext.ja va:366)
at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationContext.java:122)
at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationContext.java:66)
at org.springframework.batch.sample.AbstractBatchLaun cherTests.createApplicationContext(AbstractBatchLa uncherTests.java:53)
at org.springframework.test.AbstractSingleSpringConte xtTests.loadContextLocations(AbstractSingleSpringC ontextTests.java:212)
at org.springframework.test.AbstractSingleSpringConte xtTests.loadContext(AbstractSingleSpringContextTes ts.java:187)
at org.springframework.test.AbstractSpringContextTest s.getContext(AbstractSpringContextTests.java:140)
at org.springframework.test.AbstractSingleSpringConte xtTests.setUp(AbstractSingleSpringContextTests.jav a:100)
at junit.framework.TestCase.runBare(TestCase.java:132 )
at org.springframework.test.ConditionalTestCase.runBa re(ConditionalTestCase.java:76)
at junit.framework.TestResult$1.protect(TestResult.ja va:110)
at junit.framework.TestResult.runProtected(TestResult .java:128)
at junit.framework.TestResult.run(TestResult.java:113 )
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:2 32)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:421)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.run(RemoteTestRunner.java:305)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.main(RemoteTestRunner.java:186)
Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'simpleJobRepository' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'sqlJobDao' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'sqlJobDao' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'jobIncrementer' while setting bean property 'jobIncrementer'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'jobIncrementer' defined in class path resource [data-source-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.InvalidPropertyException : Invalid property 'columnName' of bean class [org.springframework.jdbc.support.incrementer.Oracl eSequenceMaxValueIncrementer]: No property 'columnName' found
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:274)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:104)
at org.springframework.beans.factory.support.Construc torResolver.resolveConstructorArguments(Constructo rResolver.java:447)
at org.springframework.beans.factory.support.Construc torResolver.autowireConstructor(ConstructorResolve r.java:148)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.autowireConstructor(Abs tractAutowireCapableBeanFactory.java:957)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBeanInstance(Abst ractAutowireCapableBeanFactory.java:869)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:514)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:268)
... 33 more
Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'sqlJobDao' defined in class path resource [simple-container-definition.xml]: Cannot resolve reference to bean 'jobIncrementer' while setting bean property 'jobIncrementer'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'jobIncrementer' defined in class path resource [data-source-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.InvalidPropertyException : Invalid property 'columnName' of bean class
... 47 more
Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'jobIncrementer' defined in class path resource [data-source-context.xml]: Initialization of bean failed; nested exception is org.springframework.beans.InvalidPropertyException : Invalid property 'columnName' of bean class
... 59 more
Caused by: org.springframework.beans.InvalidPropertyException : Invalid property 'columnName' of bean class
... 67 more

Has anyone tried testing the samples in Oracle? Can you guide me on where the problem lies, or I have missed anything in the process? Couldn't find much help in the forum and the web, and hence this post.

Thanks,
-Mahesh

[Dave Syer]

I'm guessing you set the batch.database.incrementer.class to something that doesn't work with Oracle. What are you using? One of the out-of-the-box incrementers might work (despite the fact that none is targeted at Oracle). You might have to write your own incrementer, or modify the data-source-context.xml (and the DDL) so that it uses an OracleSequenceMaxValueIncrementer. If you get it to work with OracleSequenceMaxValueIncrementer we would be interested in a patch (via JIRA http://opensource.atlassian.com/projects/spring/browse/BATCH-240). I'm sure someone else must be using Oracle, but most of the folks I know using Spring Batch in anger seem to be on DB2 for some reason.

I would also be interested to know what errors you were seeing with HSQLDB, as that is what we use in our CI build, so it definitely works for most people.

[lucasward]

You should be able to just comment out the column name line in batch.properties, it's only there to support HSQLDB.

[adnan1146]

I successfully run simpleTaskletJob in Oracle as well as in HSQL DB. But i got problem running tradeJob in Oracle, while it ran successfully in HSQL DB.
I ran the above mentioned jobs through JUnit test cases and also through Job Launcher.

Here is the exception i got when i tried to run tradeJob in oracle:



org.springframework.jdbc.UncategorizedSQLException : Executing query; uncategorized SQLException for SQL [SELECT id, quantity, price, customer from TRADE]; SQL state [null]; error code [17023]; Unsupported feature; nested exception is java.sql.SQLException: Unsupported feature
Caused by: java.sql.SQLException: Unsupported feature
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBE rror.java:134)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBE rror.java:179)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBE rror.java:269)
at oracle.jdbc.dbaccess.DBError.throwUnsupportedFeatu reSqlException(DBError.java:690)
at oracle.jdbc.driver.OracleConnection.createStatemen t(OracleConnection.java:3292)
at org.springframework.batch.io.cursor.JdbcCursorInpu tSource.executeQuery(JdbcCursorInputSource.java:32 4)
at org.springframework.batch.io.cursor.JdbcCursorInpu tSource.open(JdbcCursorInputSource.java:564)
at org.springframework.batch.io.cursor.JdbcCursorInpu tSource.read(JdbcCursorInputSource.java:203)
at org.springframework.batch.io.cursor.JdbcCursorInpu tSource$$FastClassByCGLIB$$762918c9.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy. java:149)
at org.springframework.aop.framework.Cglib2AopProxy$C glibMethodInvocation.invokeJoinpoint(Cglib2AopProx y.java:695)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :139)
at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.doProceed(DelegatingIntroductionIn terceptor.java:131)
at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.invoke(DelegatingIntroductionInter ceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :161)
at org.springframework.aop.framework.Cglib2AopProxy$D ynamicAdvisedInterceptor.intercept(Cglib2AopProxy. java:630)
at org.springframework.batch.io.cursor.JdbcCursorInpu tSource$$EnhancerByCGLIB$$7cd4e57c.read(<generated>)
at org.springframework.batch.item.provider.InputSourc eItemProvider.next(InputSourceItemProvider.java:48 )
at org.springframework.batch.execution.tasklet.ItemPr oviderProcessTasklet.execute(ItemProviderProcessTa sklet.java:141)
at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.doTaskletProcessing(SimpleStepExe cutor.java:372)

[lucasward]

Are you also using Oracle 10g? The code that is throwing the error is trying to open a read_only cursor. Is the supported in your database?

[adnan1146]

i am using oracle 9i.

[neetu]

I have done the changes as mentioned in the above post. I still get the following error:

java.lang.ClassCastException: java.lang.Long

[Dave Syer]

What JVM version are you using? We target 1.4 (and above) with our distributed jars. If you compiled it yourself you can figure out how to make it compatible.

[neetu]

Hi Dave,
I am using JDK1.5. The oracle jar, I used was not correct. Now its working fine.

[neetu]

The RestartFunctionalTest when executed alongwith oracle as database, it inserts 7 rows i.e. it restart from the first data. Because of this, the unit test fails.


junit.framework.AssertionFailedError: expected:<5> but was:<7>
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:2 80)
at junit.framework.Assert.assertEquals(Assert.java:64 )
at junit.framework.Assert.assertEquals(Assert.java:19 8)
at junit.framework.Assert.assertEquals(Assert.java:20 4)
at org.springframework.batch.sample.RestartFunctional Tests.testRestart(RestartFunctionalTests.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknow n Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Un known Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at junit.framework.TestCase.runTest(TestCase.java:164 )
at junit.framework.TestCase.runBare(TestCase.java:130 )
at org.springframework.test.ConditionalTestCase.runBa re(ConditionalTestCase.java:76)
at junit.framework.TestResult$1.protect(TestResult.ja va:106)
at junit.framework.TestResult.runProtected(TestResult .java:124)
at junit.framework.TestResult.run(TestResult.java:109 )
at junit.framework.TestCase.run(TestCase.java:120)
at junit.framework.TestSuite.runTest(TestSuite.java:2 30)
at junit.framework.TestSuite.run(TestSuite.java:225)
at org.eclipse.jdt.internal.junit.runner.junit3.JUnit 3TestReference.run(JUnit3TestReference.java:128)
at org.eclipse.jdt.internal.junit.runner.TestExecutio n.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.run(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.main(RemoteTestRunner.java:196)

It works absolutely fine with hsqldb.

[Dave Syer]

It works for me. What version of Spring Batch are you using?

[neetu]

I am using the samples provided with spring-batch-1.0.0.m3-with-dependencies

[gaurav]

Even I got similar Assertion Failuer message in RestartFunctionTest. I used spring-batch-1.0.0.m3-with-dependencies release version and back end is Oracle.The sample code is inserting 2 records in the database the throws execption(as desired) and the sample code restarts all over gain and the insert 5 records including previous 2 that makes it 7(whereas expected value is 5 i.e. starting from 3rd record till 5th record).
Any suggestion why it is behave in this manner.
I would be glad if I can get location in the sample code where the batch framework campare the no. of records inserted(commited) and restart from next record.

[Dave Syer]

It seems there is a problem with the way the restart data is being persisted. Can someone confirm that (like me) it actually works with 10g in some cases (I just downloaded the XE edition and it worked out of the box)? Can someone with a test failure also get us some more information about the restart data in between runs (it is stored in the batch_step table in plain text form)? Could there be some unusual set up where varchar columns have limited sizes smaller than required? I'm struggling to think of why it would work for me and not someone else.

[gaurav]

Hello,
In reference with the above problem, I would like to provide the following input :

Oracle JARs used : classes12.jar and ojdbc14.jar

According to me, the problem is due to the cursor not maintaining the value of the last fetch record rather I would say JARs used might be incompatible to work with cursor properly(assumption) or there has to be some configuration done in my Oracle10g database.

[Dave Syer]

Right, so you'd better check the version of Oracle that the jars come from. If it isn't 10g (and reasonably high patch level) you probably need to upgrade.

[gaurav]

I have used the Oracle jar compatiple with 10g and its version is 10.2.

[Dominic Aitken]

I'm sure someone else must be using Oracle, but most of the folks I know using Spring Batch in anger seem to be on DB2 for some reason.

I have written a restartable custom batch process using spring batch (m3), and am testing it using both Derby and Oracle (9i) as databases to hold the spring batch job control tables. I get no problems with Derby, but have the following issues with Oracle which may be relevant here:

1. I used a SimpleJobIdentifier. When accessing the BATCH_JOB table spring batch sets the value of JOB_KEY to an empty string. (Set log4j.category.org.springframework.jdbc=DEBUG to see this). On Oracle a zero length string is treated as a NULL so on restart of a FAILED job the effective predicate in SQL to locate data on BATCH_JOB becomes <... JOB_KEY = NULL ... > where JOB_KEY data is NULL. As NULLs do not equal on SQL no row is returned and the job proceeds as if it were a new run and not a recovery of a failed run. I worked round this by using ScheduledJobIdentifier and specifying a key, but probably spring batch should either place a dummy value in JOB_KEY when one is not specified (easy) or not use JOB_KEY as a predicate in SQL commands where JOB_KEY was not specified (logical, but more complex).

2. I also found a difference in the use of DATE for column SCHEDULE_DATE. If you set up the "Job" with a date/time value in the jobId e.g.
jobId.setScheduleDate(new Date()); then Derby ignores the time portion, but Oracle DATE columns store it and so on recovery having a different time causes the job to fail to locate the row on BATCH_JOB. If spring batch only expects to use the date portion of the the ScheduleDate, then it might be worthwhile for it to internally set the time part to zero to avoid this discrepency. Although I accept it is easy for the caller to ensure this criterion is met.

I'm not sure if any of this helps in this thread, but perhaps it may have some bearing on it.

[Dave Syer]

All this is really useful information (so I created an issue: http://jira.springframework.org/browse/BATCH-281), and some new tests obviously need to be added. But does it explain why some people have reported failed unit tests? The tests all pass in my environment against Derby and 10g, so can anyone shed any light on that?

[paulaqian]

I am using 10.2.0.3 oracle driver but I am accessing an 9i database. And I still get junit.framework.AssertionFailedError: expected:<5> but was:<7>:

[cerfrgrrrl]

I am also using the 10.2.0.3 oracle accessing 10g database, same error with m5.

[Dave Syer]

Can you provide some more data? It would help a lot if we could see the job meta data. Best would be if you would: start with a clean database; run only the sample test that fails; post back the contents of the step execution and execution context tables.

Track progress here: http://jira.springframework.org/browse/BATCH-439

[paulaqian]

I also had the same problem with m5 here. Trace attached.