Beginning with Spring batch

[Seph]

Hello everybody,

First of all, let me introduce myself as it is my first post here. I'm Patrick, working for an insurance company as software developer. I lately was involved in a project for migrating some COBOL application into java. This migration needs to be a functional copy of the previous application without any changes and deserve only a technical purpose : we soon won't have any competency left for COBOL.

That being said, as you probably know, COBOL applications were really much focuses on jobs, steps, in a word "batchs". Then... well... I need to have somehow a way of making java batchs. Quick look on internet, guess what, Spring batch seems to be great for it as I would need to handle transactions to ensure either the complete batch succeed or nothing is done, restarting / stopping batchs, monitoring, ...

Now, I am JSF / JAX-WS developer with application layer done through JPA / hibernate, EJB3... but never touched to the Spring Framework.

I tried to work with the documentation at this address, but I cannot find much information focused on installation. The concepts are rather focused to the batch possibilities.

As a humble beginning, I'd like just to run a first batch, calling nothing more than a small Reader / Writer. I do have an EJB Eclipse project called testbatch, which has an EJB source folder called ejbModule. In it, just a simple package test.batch containing these files :

TestReader.java :

public class TestReader implements ItemReader<String> {
	static String[] strings = {"test1", "test2", "test3", "test4"};
	static Integer index = 0;

	@Override
	public String read() throws Exception, UnexpectedInputException, ParseException {
		if(index <= 3){
			String s = strings[index];
			index++;
			return s;
		}
		return null;
	}
}

TestWriter.java :

public class TestWriter implements ItemWriter<String> {

	@Override
	public void write(List<? extends String> items) throws Exception {
		for(String s: items) System.out.println(s);
	}

}

batch-sample-context.xml :

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

	<bean id="TestReader" class="test.batch.TestReader" />
	<bean id="TestWriter" class="test.batch.TestWriter" />
	
	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
	</bean>

	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="transactionManager" ref="transactionManager" />
	</bean>

	<bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" >
	</bean>

	<bean id="minimal"
		class="org.springframework.batch.core.job.SimpleJob">
		<property name="jobRepository" ref="jobRepository" />
		<property name="steps">
			<bean id="simpleStep"
				class="org.springframework.batch.core.step.item.SimpleStepFactoryBean">
				<property name="transactionManager" ref="transactionManager" />
				<property name="jobRepository" ref="jobRepository" />
				<property name="itemReader" ref="TestReader" />
				<property name="itemWriter" ref="TestWriter" />
			</bean>
		</property>
	</bean>

</beans>

I found samples of these simple calls at this address (sorry it's in french).

When I make a configuration in Eclipse with

org.springframework.batch.core.launch.support.CommandLineJobRunner

as main class and

batch-sample-context.xml minimal

as program arguments, I always receive this error :

log4j:WARN No appenders could be found for logger (org.springframework.context.support.ClassPathXmlApplicationContext).
log4j:WARN Please initialize the log4j system properly.

Whatever I'm trying, I just cannot have log4j be configured properly. I found on internet I should try something like adding this to my batch xml file :

	<bean id="log4jInitialisation" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
		<property name="targetClass"
			value="org.springframework.util.Log4jConfigurer" />
		<property name="targetMethod" value="initLogging" />
		<property name="arguments">
			<list>
				<value>ejbModule/test/batch/log4j.xml</value>
			</list>
		</property>
	</bean>

I tried it with this xml :

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
  <appender name="console" class="org.apache.log4j.ConsoleAppender"> 
    <param name="Target" value="System.out"/> 
    <layout class="org.apache.log4j.PatternLayout"> 
      <param name="ConversionPattern" value="%-5p %c{1} - %m%n"/> 
    </layout> 
  </appender> 

  <root> 
    <priority value ="debug" /> 
    <appender-ref ref="console" /> 
  </root>
  
</log4j:configuration>

but wasn't able to make it work.

This example is really stupid, but making a few Hello World to just install the environment is already a big part of the deal.

Do you have advices to make my stupid test work ?

Here it is my first test, which I couldn't make work, but it will eventually be for calling EJB's of my application layer embedded in the JBoss 5.1 Application Server, with transaction management,.... If there exist some good installation guides on Spring Batch, I would fairly appreciate as I could not find anything on this topic in the official documentation.

Another additional point : do I need to jump into Spring Framework to understand how to configure and develop with the Spring Batch facilities ?

Thank you in advance for your advices... integration of new libraries / frameworks are not easy in the Java world, but I hardly found so few documentation for an easy and just simple beginning. I found much, I tested much, I wasn't able to make a single sample work

[dma_k]

Hello everybody,

As a humble beginning, I'd like just to run a first batch, calling nothing more than a small Reader / Writer. I do have an EJB Eclipse project called testbatch, which has an EJB source folder called ejbModule. In it, just a simple package test.batch containing these files :

TestReader.java :
TestWriter.java :

Those files look good.

batch-sample-context.xml :

	<bean id="minimal"
		class="org.springframework.batch.core.job.SimpleJob">
		<property name="jobRepository" ref="jobRepository" />
		<property name="steps">
			<bean id="simpleStep"
				class="org.springframework.batch.core.step.item.SimpleStepFactoryBean">
				<property name="transactionManager" ref="transactionManager" />
				<property name="jobRepository" ref="jobRepository" />
				<property name="itemReader" ref="TestReader" />
				<property name="itemWriter" ref="TestWriter" />
			</bean>
		</property>
	</bean>

Instead of writing a lot of XML like this, I would suggest you to use "batch" namespace as shown here.

log4j:WARN No appenders could be found for logger (org.springframework.context.support.ClassPathXmlApplicationContext).
log4j:WARN Please initialize the log4j system properly.

I suppose you cant initialize Log4J from Spring context, while Spring starts using it before the context is created (chicken and egg ). So you need to put your log4j.properties file into your resources root or alternatively use org.apache.log4j.xml.DOMConfigurator.doConfigure() + XML you provided before creating the Spring context; but as you want to to CommandLineJobRunner that last is not an option for you.

Do you have advices to make my stupid test work ?

A bit problematic to advise you without logs. Perhaps when you make logging working, everything will become clearer.

[Seph]

Hi and thank you very much for the fast answer.

I needed a few tests to know where I should place my log4j.properties (in fact I just added it in every folder just to see if I could make it work... when seeing it was the case, I just removed one by one the files to discover the one enabling it). The right place is apparently the root directory of the source folder (the one called ejbModule in my project). Here is my simple file :

log4j.rootCategory=INFO, S
log4j.appender.S=org.apache.log4j.ConsoleAppender
log4j.appender.S.layout=org.apache.log4j.PatternLayout
log4j.appender.S.layout.ConversionPattern=%d %-5p [%c] (%t) %m%n

I also moved my batch-sample-context.xml a bit for this :

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:aop="http://www.springframework.org/schema/aop"
	xmlns:p="http://www.springframework.org/schema/p"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop.xsd
		http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
		
	<bean id="PoliceReader" class="ch.mobi.testprest.batch.PoliceReader"></bean>
	<bean id="PoliceWriter" class="ch.mobi.testprest.batch.PoliceWriter"></bean>
	
	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
	</bean>
	
	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="transactionManager" ref="transactionManager" />
	</bean>
	
	<bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" >
	</bean>

	<bean id="minimal"
		class="org.springframework.batch.core.job.SimpleJob">
		<property name="jobRepository" ref="jobRepository" />
		<property name="steps">
			<bean id="simpleStep"
				class="org.springframework.batch.core.step.item.SimpleStepFactoryBean">
				<property name="transactionManager" ref="transactionManager" />
				<property name="jobRepository" ref="jobRepository" />
				<property name="itemReader" ref="PoliceReader" />
				<property name="itemWriter" ref="PoliceWriter" />
			</bean>
		</property>
	</bean>

</beans>

And... magic... it works. Well, almost, I had to correct dozens of Java Exceptions, but they were pretty clear once the log4j configuration is enabled to read what Java want to say (and it's a very talkative friend).

Now, I will need to communicate with my JBoss Server to connect to the Application EJB's. Does someone have a link pointing to a tutorial explaining how to call application server components from Spring Batch ? I dit not get a single hint to a configuration in all I read on this topic (documentation do not contain "ejb" keyword, or "enterprise" or any term related to application server, I really wonder where are the installation and configuration documentations).

[dma_k]

Now, I will need to communicate with my JBoss Server to connect to the Application EJB's. Does someone have a link pointing to a tutorial explaining how to call application server components from Spring Batch ?

If you know how EJBs work, this note about EBJ-to-Spring integration will be helpful. JTA transactions are also supported.

I dit not get a single hint to a configuration in all I read on this topic (documentation do not contain "ejb" keyword, or "enterprise" or any term related to application server, I really wonder where are the installation and configuration documentations).

Spring Batch cannot integrate all the technologies around the world: there are separate Spring modules or subprojects for that. Try googling for "Spring and EJB"

[Seph]

Thank you for the notice for connecting Spring to EJB's. As I mentioned, I never used Spring framework before trying to use Spring Batch. Then this part of the config was quite obscure for me.

I'll come back if I do have further questions.