a) 1. Without knowing too much about the number of files, etc, launching a job per file may be ok. The alternative would be to batch the messages together via aggregation before passing them to your job. In either case, I would use the message based job launching stuff available in spring batch integration.
a) 2. There is a JobLaunchingMessageHandler within spring batch integration that takes a JobLaunchRequest message. I would expect you to use this to launch the jobs. Basically creating a message transformer to transform your file requests into JobLaunchRequest messages.
b) The message you will pass will be a JobLaunchRequest which consists of a Job and a JobParameters object.
c) The JobLaunchingMessageHandler will return the JobExecution via the configured channel once the execution is complete (assuming synchronous execution of the job).
Below is an example of launching a job via a message in action (it doesn't handle the message transformation mentioned above). The JobRepository and JobLauncher have been removed for brevity but they are standard configurations.
Code:
<int:annotation-config/>
<int:channel id="job-requests"/>
<int:channel id="job-responses">
<int:queue/>
</int:channel>
<int:service-activator input-channel="job-requests">
<bean class="org.springframework.batch.integration.launch.JobLaunchingMessageHandler">
<constructor-arg ref="jobLauncher"/>
</bean>
</int:service-activator>
<bean id="taskletAdapter" class="org.springframework.batch.core.step.tasklet.MethodInvokingTaskletAdapter">
<property name="targetObject">
<bean class="java.lang.String">
<constructor-arg value="Invoked via message"/>
</bean>
</property>
<property name="targetMethod" value="length"/>
</bean>
<batch:job id="simpleJob">
<batch:step id="simpleJob.step1">
<batch:tasklet ref="taskletAdapter"/>
</batch:step>
</batch:job>
Code:
import static org.junit.Assert.assertEquals;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.integration.launch.JobLaunchRequest;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.integration.Message;
import org.springframework.integration.MessageChannel;
import org.springframework.integration.MessageHeaders;
import org.springframework.integration.core.PollableChannel;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class MessageTriggerTests {
@Autowired
private Job job;
@Autowired
@Qualifier("job-requests")
private MessageChannel jobRequestChannel;
@Autowired
@Qualifier("job-responses")
private PollableChannel jobResponseChannel;
@Before
public void setUp() throws Exception {
}
@SuppressWarnings("unchecked")
@Test
public void test() {
JobLaunchRequest request = new JobLaunchRequest(job, new JobParameters());
jobRequestChannel.send(MessageBuilder.withPayload(request).setHeader(MessageHeaders.REPLY_CHANNEL, jobResponseChannel).build());
Message<JobExecution> results = (Message<JobExecution>) jobResponseChannel.receive(10l);
assertEquals(BatchStatus.COMPLETED, results.getPayload().getStatus());
}
}
With regards to the MongoDB aspect, are you implying that you want to use MongoDB for a JobRepository implementation? If so, that may be a bit problematic. MongoDB is not transactional so using it for a repository, while possible, would not be a sound option IMHO. You can take a look at the MongoItemWriter I committed today in the git repository to see what we are doing to emulate transactionality, but all it really does is push off until the last possible point the writing of the data to that repository (similar to how we handle transactions with files).
non-stop batch engine
Reply With Quote
