Spring Batch Transactions in J2EE container

Hi all :)

I am validating the integration of Spring Batch in our SOA architecture. The goal is that batch programs and online applications use the same business services. These services are implemented as EJBs stateless session beans 3.1 and run on JBoss application server 7.1.
We can have two types of jobs which follow the following designs:
- extraction job: reads from a DB (direct through JDBC), eventually some processing, and XML writing.
- processing job: reads from an XML file, processing (through EJB services) and XML writing

There are several business databases and one central database for spring batch metadata.

I plan to embed the jobs directly in the same J2EE container as EJB services, in order to use the same transaction manager for all the process (JBoss TS), and to optimize performances by not using EJB remote client.
More exactly, the jobs will be launch inside an EJB SLSB component in the same EAR file that business services (so we can work by references).

My interrogations are essentially around transaction aspects:
I see that in a chunk oriented step, with JDBC cursor reader, SB creates a first transaction for initializing step data and then a second one for the chunk itself.

1- why reading needs a transaction since data are just read?

2- There are two different databases (one for SB metadata, one for JDBC cursor Reader) and two different transactions, do we have to use XA transactions? If yes, what is the spring configuration? Declaring a bean JTA transaction manager in Jobs configuration is enough? If no, what is the correct configuration?

3- I saw that we can embed JDBC cursor reader transaction in the same transaction as the chunk one, what is the interest? In this case, I guess it is mandatory to use XA transactions?

4- the Job repository is associated with a transaction manager. If we specify an other transaction manager at the tasklet level, which one is used by the JobRepository to access SB metadata database? What is the correct way to process in my use case?

5- All the SB features are available when using it in a J2EE container? For instance, regarding scaling patterns is it possible to use multithreading knowing that we can call a EJB in the processor of a Step? What about remote scaling?

6- We plan to launch the Jobs through a Scheduler. Since we are in an asyncrhonous way, what is the pattern you recommend for the scheduler to know about Jobs states, ends and so on? We cannot in this case using Jobs exit codes.

Thanks in advance for your responses!

Quote:

---

Originally Posted by benmasi

1- why reading needs a transaction since data are just read?

---

Reading does not require a transaction. The chunk requires a transaction but the read itself does not.

Quote:

---

2- There are two different databases (one for SB metadata, one for JDBC cursor Reader) and two different transactions, do we have to use XA transactions? If yes, what is the spring configuration? Declaring a bean JTA transaction manager in Jobs configuration is enough? If no, what is the correct configuration?

---

Yes XA is required if you are using two different datasources. Your job should already have a transaction manager configured. Replace that bean with your JTA transaction manager (named the same) and everything should be injected fine. Otherwise, you'll have to inject it manually into a few different spots.

Quote:

---

3- I saw that we can embed JDBC cursor reader transaction in the same transaction as the chunk one, what is the interest? In this case, I guess it is mandatory to use XA transactions?

---

I'm not following you with regards to the reader's transaction...no transaction is created by the reader itself. It participates in the chunk's transaction.

Quote:

---

4- the Job repository is associated with a transaction manager. If we specify an other transaction manager at the tasklet level, which one is used by the JobRepository to access SB metadata database? What is the correct way to process in my use case?

---

You should only use one transaction manager. Hence the need for XA if you are going to use multiple data sources.

Quote:

---

5- All the SB features are available when using it in a J2EE container? For instance, regarding scaling patterns is it possible to use multithreading knowing that we can call a EJB in the processor of a Step? What about remote scaling?

---

Yes. We use Spring's components for things like multithreading (using a TaskExecutor) so there is no dependance or hinderance by using a container.

Quote:

---

6- We plan to launch the Jobs through a Scheduler. Since we are in an asyncrhonous way, what is the pattern you recommend for the scheduler to know about Jobs states, ends and so on? We cannot in this case using Jobs exit codes.

---

If you are launching the jobs asynchronously, the easiest way to determine the state of the job is to poll the job repository for the state.

Hi,

Quote:

---

Reading does not require a transaction. The chunk requires a transaction but the read itself does not.

---

Quote:

---

I'm not following you with regards to the reader's transaction...no transaction is created by the reader itself. It participates in the chunk's transaction.

---

It is said in the javadoc of the class AbstractCursorItemReader (mother class of JDBCCursorItemReader) that :
- "By default the cursor will be opened using a separate connection. The ResultSet for the cursor is held open regardless of commits or roll backs in a surrounding transaction."
- "There is an option (setUseSharedExtendedConnection(boolean) that will share the connection used for the cursor with the rest of the step processing ..."

So there are well two different transactions: one for the Chunk, and an other created by the JDBCCursorItemReader, no? So using XA transactions is not necessary, no? Or there is something that I do not catch well in the transaction mangement of Spring Batch? :confused:

Thanks for your answer.

No, using XA is not required just to use the JdbcCursorItemReader. Keep in mind, that the cursor will be opened and maintained open outside of what happens with the transaction. You typically won't want that read to be part of the transaction since it is only reading (no changing of data occurs via the reader).

So, i am still confused. You answered my first post i need XA, and the second one that not. :confused:

Maybe i am not clear, and if that is the case, i am sorry. I am going to re-explain clearly my use case :
- my Job is running on an application server embedded in a EJB 3.1 stateless application (i dont think that change anything)
- the Job uses a database to store SB metadata (db1)
- the Job is composed of a unique Step, which uses a JDBCCursorItemReader which reads business data from an other database (db2).

Do I need to use XA transactions in that case?

Sorry for the confusion. XA is needed if you are using multiple data sources. So if you have your batch schema in one database and your business processing is occurring in another database, then you need XA.

So in your example, you are using two databases so you need XA. My previous comment saying that XA is not required referred to the idea that XA was required to use the JdbcCursorItemReader. This is not true. The reason you need XA is because the business processing is against a different database than the SB schema.

Hi!

So, if I resume.

1- in the following configuration:
– one Job with one Step
– one database for Spring Batch metadata (db1)
– the Step has a JDBCCursorItemReader which reads from business data from an other database (db2)
– the Step has a FlatFileItemWriter

I understand that I do not need XA transactions, right? (since only one database (db1) is used by the chunk transaction)

2- and in this configuration:
– one Job with one Step
– one database for Spring Batch metadata (db1)
–the Step has a JDBCCursorItemReader which reads from business data from an other database (db2)
– the Step writes the outputs (business data) to a third database (db3)

In this case, I need XA transactions, is it right? (since two databases (db1 and db3) are in the chunk transaction)

You are correct.