Concurrent Steps and Readers that share a transaction

[matt.friedman]

Hi,

I'm new to spring batch. I am trying to formulate a way to do the following.

I have the first driving query Q(a). The result set of Q(a) will drive two dependent queries Q(b) and Q(c). That is, the where clause of Q(b) and Q(c) depend on the result set of Q(a).

I would like to first execute Q(a) and read a set of results. After that I'd like to issue the two dependent queries concurrently. And, I want all of this to happen in one transaction. That is, all of Q(a), Q(b), and Q(c) should happen in the same transaction.

Is this even possible with spring batch?

So far, I suppose Q(a) can happen in a step. After that I can use <flow> to run both of the dependent queries.

Any hints and / or direction would be greatly appreciated.

Thanks,
Matt

[mminella]

To get all of that in a single transaction, it needs to be in a single step (transactions do not span steps in Spring Batch). When you say

The result set of Q(a) will drive two dependent queries Q(b) and Q(c).

does that mean that for each item in the result set of Q(a) you want to execute Q(b) and Q(c) or based on the aggregated results of Q(a), execute Q(b) and Q(c) once?

[matt.friedman]

From Q(a) I will get the lowest and highest id and use those to constrain the queries for Q(b) and Q(c).

(i.e. where lowest <= id <= highest, or something like that)

BTW, I've given up on the parallel execution because I don't think the queries will run in parallel on the same database connection anyway.

I've moved forward on the task so far using Spring Integration, especially the aggregator and json transformer.

Eventually, I think I can convert the query procedures into an ItemReader which would allow me to use a single step as you recommend. For now, I'm executing the queries and writing the results into a spring integration gateway.

Ideally, I'll convert this to spring batch; To do so, I still need to learn how to use spring integration and batch together.

From what I can tell so far I can invoke the spring integration flow from an ItemProcessor, but it still isn't clear to me how to get the results back from SI and then pass those results to the ItemWriter. Workin on it!

Any thoughts are greatly appreciated.

Thanks,
Matt