jdbccursoritemreader issue

[Dave Syer]

I'm interested in what you mean exactly by "never able to process more than 1500 records". And were you actually using a multi-threaded step to read from a cursor? It has no right to work (and is probably broken for restart, so be very careful).

[joecarew]

I jumped the gun that the driver change fixed the problem. I went to a faster system and the problem came back at exactly the same step. I'm attaching the (sanitized) job config. Is there another class we should use instead of JdbcCursorItemReader?

Here is also a little bit more context of the batch job. The first step runs a process that populates a table. The second step reads that table and processes it's records in chunks. The rest of the steps read from the output of the previous step, execute a service and write to another table.

[Dave Syer]

The problem is that you can't use a JdbcCursorItemReader with a multi-threaded task executor. Look at the difference between StagingItemReader and JdbcCursorItemReader to see why. You either need to make your first stagingInitializationReader step single-threaded or use a different (custom) reader.

[jfreed]

Thanks for the suggestion. We created a custom multi-threaded reader, and that did the trick.

[Med Dab]

Hello
I'm using spring batch 1.1.4.RELEASE
I have the unexpected cursor position exception when I'm using the ThreadPoolTaskExecutor taskExecutor in my step. If I desable verifyCursorPosition, sometimes two or three threads insert the same record. How can'I use spring batch to solve this problem? Or I must make a synchronize on my cursor to be sur that only a single thread read the cursor?

[wasanTH]

Dear all
please kindly help me solve the problem cursor position change
I have problem same 'cch'
about org.springframework.dao.InvalidDataAccessResourceU sageException: Unexpected cursor position change

I need parallel step

this problem from

Job :
<bean id="taskExecutorAsync" class="org.springframework.core.task.SimpleAsyncTa skExecutor" />


<job id="TestJob">
<step id="step1" next="step2">
<tasklet>
<chunk
reader="readerStep1"
writer="writerStep2"
commit-interval="100"/>
</tasklet>
</step>
<step id="step2" next="step3">
<tasklet>
<chunk
reader="readerStep2"
writer="writerStep2"
commit-interval="100"
task-executor="taskExecutorAsync"/>
</tasklet>
</step>
<step id="step3">
<tasklet>
<chunk
reader="readerStep3"
writer="writerStep3"
commit-interval="100"
task-executor="taskExecutorAsync"/>
</tasklet>
</step>
</job>

oracle xa :
<datasources>
<xa-datasource>
<jndi-name>DS_XAOracleDS</jndi-name>
<!-- uncomment to enable interleaving <interleaving/> -->
<isSameRM-override-value>false</isSameRM-override-value>
<xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
<xa-datasource-property name="URL">jdbc:oracle:thin:@localhost:1521:xe</xa-datasource-property>
<xa-datasource-property name="User">test</xa-datasource-property>
<xa-datasource-property name="Password">test</xa-datasource-property>
<!-- Uses the pingDatabase method to check a connection is still valid before handing it out from the pool -->
<!--valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.Oracle ValidConnectionChecker</valid-connection-checker-class-name-->
<!-- Checks the Oracle error codes and messages for fatal errors -->
<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.Oracle ExceptionSorter</exception-sorter-class-name>
<!-- Oracles XA datasource cannot reuse a connection outside a transaction once enlisted in a global transaction and vice-versa -->
<no-tx-separate-pools/>

<!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml (optional) -->
<metadata>
<type-mapping>Oracle9i</type-mapping>
</metadata>
</xa-datasource>

Env : Oracle 10g ,SpringBatch 2.0

Thank you

[wasanTH]

I can resolve problem about cursor position change can see information below

5.3.5. Split Flows
Every scenario described so far has involved a Job that executes its Steps one at a time in a linear
fashion. In addition to this typical style, the Spring Batch namespace also allows for a job to be
configured with parallel flows using the 'split' element. As is seen below, the 'split' element contains
one or more 'flow' elements, where entire separate flows can be defined. A 'split' element may also
contain any of the previously discussed transition elements such as the 'next' attribute or the 'next',
'end', 'fail', or 'pause' elements.
<split id="split1" next="step4">
<flow>
<step id="step1" parent="s1" next="step2"/>
<step id="step2" parent="s2"/>
</flow>
<flow>
<step id="step3" parent="s3"/>
</flow>
</split>
<step id="step4" parent="s4"/>

[MmarcoM]

You are trying to use a TaskExecutorRepeatTemplate with a cursor reader? Maybe that's the problem (two threads access the cursor simultaneously). Can you post the configuration and explain your use case a bit?

hi, sorry for hijacking this thread

i am having same problem

- reading 40K rows table
- do some processign with each item (call a service which will retrieve extra data)
- output the data to a file

i am getting the Cursor exception... any idea on how can i get around it?

the expensive logic is done in processor.....

w/kindest regards
marco