Not Serializable Problem

[nmgo]

Hi all,

I'm currently trying Spring Batch 2.0.0.RC1 to run a job with Quartz.
I have a very simple sample job, that when I'm using an In-Memory Repository works fine, but when I change jobRepository to a persistent one gives the following error:

Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.scheduling.quartz.SchedulerFactoryBean#0' defined in class path resource [applicationContext.xml]: Invocation of init method failed; nested exception is org.quartz.JobPersistenceException: Couldn't store job: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.SimpleJobLauncher [See nested exception: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.SimpleJobLauncher]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1338)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:423)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
	at eu.europa.emsa.BatchQuartzTest.main(BatchQuartzTest.java:88)
Caused by: org.quartz.JobPersistenceException: Couldn't store job: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.SimpleJobLauncher [See nested exception: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.SimpleJobLauncher]
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1041)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$4.execute(JobStoreSupport.java:1011)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$39.execute(JobStoreSupport.java:3590)
	at org.quartz.impl.jdbcjobstore.JobStoreCMT.executeInLock(JobStoreCMT.java:244)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInLock(JobStoreSupport.java:3586)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1007)
	at org.quartz.core.QuartzScheduler.addJob(QuartzScheduler.java:785)
	at org.quartz.impl.StdScheduler.addJob(StdScheduler.java:278)
	at org.springframework.scheduling.quartz.SchedulerAccessor.addJobToScheduler(SchedulerAccessor.java:317)
	at org.springframework.scheduling.quartz.SchedulerAccessor.addTriggerToScheduler(SchedulerAccessor.java:340)
	at org.springframework.scheduling.quartz.SchedulerAccessor.registerJobsAndTriggers(SchedulerAccessor.java:276)
	at org.springframework.scheduling.quartz.SchedulerFactoryBean.afterPropertiesSet(SchedulerFactoryBean.java:483)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1369)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1335)
	... 15 more
Caused by: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.SimpleJobLauncher
	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.serializeJobData(StdJDBCDelegate.java:3354)
	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.insertJobDetail(StdJDBCDelegate.java:515)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1038)
	... 28 more

Here's my configuration file:

	<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
		<property name="driverClassName" value="com.mysql.jdbc.Driver" />
		<property name="url" value="jdbc:mysql://localhost/batch" />
		<property name="username" value="root" />
		<property name="password" value="" />
	</bean>

	<bean id="transactionManager"
		class="org.springframework.jdbc.datasource.DataSourceTransactionManager"
		lazy-init="true">
		<property name="dataSource" ref="dataSource" />
	</bean>

<batch:job-repository id="jobRepository" />

	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
		<property name="taskExecutor">
			<bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
		</property>
	</bean>

	<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
		<property name="triggers">
			<bean id="cronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
				<property name="jobDetail" ref="jobDetail" />
				<property name="cronExpression" value="0/10 * * * * ?" />
			</bean>
		</property>
	</bean>

	<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
		<property name="jobClass" value="sample.JobLauncherDetails" />
		<property name="group" value="quartz-batch" />
		<property name="jobDataMap">
			<bean class="org.quartz.JobDataMap">
				<constructor-arg>
					<map>
						<entry key="jobName" value="myJob" />
						<entry key="jobLocator" value-ref="jobRegistry" />
						<entry key="jobLauncher" value-ref="jobLauncher" />
					</map>
				</constructor-arg>
			</bean>
		</property>
	</bean>


	<bean id="jobRegistry"
		class="org.springframework.batch.core.configuration.support.MapJobRegistry" />



	<bean
		class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
		<property name="jobRegistry" ref="jobRegistry" />
	</bean>

Any guess on what I'm doing wrong here?

Thanks!!

[Dave Syer]

The JobLauncher (and JobRegistry probably) are services, and are not intended to be Serializable. If you want to use Quartz like this you'll have to figure out a way to re-hydrate the JobDetails from something Serializable (e.g. the XML config file location).

On the other hand, I'm not sure what value your get from the Quartz job persistence. Why do you need it?

[nmgo]

Hi,

Thanks for your quick answer.
I'm new in this Spring world, and apparently I made a mistake in my application context file. When I removed default-autowire="byName" option, all worked fine.

Anyway, and answering to your question, I really don't need quartz persistence.

Thanks!!

[sabya1232000]

Hello Everybody,
I am trying to run a spring batch job from quartz cluster. To use Quartz in a cluster mode it is needed to maintain a persistent storage or Datasource. Spring Batch is able to run correctly with org.springframework.scheduling.quartz.SchedulerFac toryBean when in memory repository for quartz is used, but, the configuration load fails when a data source for quartz is introduced. The error refers to a serialization issue as below

Caused by: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'jobLauncher' is not serializable: org.springframework.batch.core.launch.support.Simp leJobLauncher
at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.seria lizeJobData(StdJDBCDelegate.java:3358)
at org.quartz.impl.jdbcjobstore.oracle.OracleDelegate .insertJobDetail(OracleDelegate.java:173)
at org.quartz.impl.jdbcjobstore.JobStoreSupport.store Job(JobStoreSupport.java:1103)


Can somebody tell me how to overcome this problem?
-------------------------------------------------------------------------
configuration usede for quartz SchedulerFactoryBean:
---------------------------------------------------------------------------

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aop="http://www.springframework.org/schema/aop"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schem...-beans-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">

<import resource="job-simple1.xml"/>

<bean class="org.springframework.scheduling.quartz.Sched ulerFactoryBean">
<property name="triggers">
<bean id="cronTrigger2" class="org.springframework.scheduling.quartz.CronT riggerBean">
<property name="jobDetail" ref="jobDetail2" />
<property name="cronExpression" value="0/60 * * * * ?" />
</bean>
</property>
<!-- This portion is needed to avail a persistent quartz cluster DB storage-->
<property name="quartzProperties">
<props>
<prop key="org.quartz.scheduler.instanceName">myClustere dScheduler</prop>
<prop key="org.quartz.scheduler.instanceId">AUTO</prop>
<prop key="org.quartz.jobStore.isClustered">true</prop>
<prop key="org.quartz.jobStore.class">org.quartz.impl.jd bcjobstore.JobStoreTX</prop>
<prop key="org.quartz.jobStore.tablePrefix">QRTZ_</prop>
<prop key="org.quartz.jobStore.driverDelegateClass">org. quartz.impl.jdbcjobstore.oracle.OracleDelegate</prop>
</props>
</property>
<property name="dataSource" ref="bdbDataSource"/>
<property name="transactionManager" ref="jtaTransactionManager"/>
<property name="nonTransactionalDataSource" ref="bdbDataSource"/>
<property name="overwriteExistingJobs" value="true"/>
<!-- End of persistent quartz cluster DB storage-->
</bean>

<bean id="jobDetail2" class="org.springframework.scheduling.quartz.JobDe tailBean">
<property name="jobClass" value="com.ppp.saby.batch.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value-ref="multilineJob"/>
<entry key="jobLocator" value-ref="jobRegistry"/>
<entry key="jobLauncher" value-ref="jobLauncher"/>
</map>
</property>
</bean>


<bean id="jobRegistry" class="org.springframework.batch.core.configuratio n.support.MapJobRegistry" />

<bean id="jobLauncher" class="org.springframework.batch.core.launch.suppo rt.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor">
<bean class="org.springframework.core.task.SimpleAsyncTa skExecutor" />
</property>
</bean>

</beans>
--------------------------------------------------
Job details form job-simple1.xml
-----------------------------------------------
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate" >
<property name="dataSource" ref="bdbDataSource" />
</bean>

<bean id="jobRepository" class="org.springframework.batch.core.repository.s upport.JobRepositoryFactoryBean">
<property name="dataSource" ref="bdbDataSource" />
<property name="transactionManager" ref="transactionManager" />
<property name="databaseType" value="oracle" />
</bean>


<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSou rceTransactionManager">
<property name="dataSource" ref="bdbDataSource" />
</bean>


<batch:job id="multilineJob">
<batch:step id="step1">
<batch:tasklet ref="helloTask">
</batch:tasklet>
</batch:step>
</batch:job>

<!-- Step1 - print hello world -->
<bean id="helloTask" class="com.batch.simpletask.HelloTask">
<property name="taskStartMessage" value="Hello World - the time is now " />
</bean>

--------------------------------------------------
Java class com.ppp.saby.batch.JobLauncherDetails
-------------------------

public class JobLauncherDetails extends QuartzJobBean{

private static Logger log = Logger.getLogger(JobLauncherDetails.class);

private Job job;

private JobLocator jobLocator;

private JobLauncher jobLauncher;

public void setJobLocator(JobLocator jobLocator) {
this.jobLocator = jobLocator;
}

public void setJobLauncher(JobLauncher jobLauncher) {
this.jobLauncher = jobLauncher;
}

public void setJobName(Job job) {
this.job = job;
}


@Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {

JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();

Calendar aDate = new GregorianCalendar();
jobParametersBuilder.addString("dateTime",String.v alueOf(aDate.getTime()));

try {
jobLauncher.run(job, jobParametersBuilder.toJobParameters());
} catch (Exception e) {
}
}

}
---------------------------------------------------------
Can somebody tell me how to overcome this problem?
If some glue code is required to allow the jobLauncher to be serialise, then please provide some example.

Thanks & Regards,
Saby

[rainman]

the jobDataMap has been configured with your jobLauncher and registry which are not serializable. When you look at a job persistent store Quartz sort of serializes the elements of the jobDataMap and stores em up into the underlying Quartz tables as BLOB columns i guess.
If your need is to only to access the jobLauncher you should be using the
schedulerContextMap inside the SchedulerFactoryBean. Since you are already extending the quartzJobBean you will not have any changes in your JobLauncherDetails since the quartzJobBean uses both the mergedMap and the scheduler's context map to set the job properties.

[lhale]

rainman:

It's all well & good that the MethodInvokerJob can't be serialized, but I bring up two observations here:

1) The original posters are asking for some kind of workaround
2) This problem affects many and has been hanging around for years

So, to cut to the chase, I found a working, tested workaround @ http://www.lucianofiandesio.com/javatales/qtzfuse.html

[rainman]

lhale:

are you suggesting the use of the applicationContext inside the job?
well that is one way..
IMHO if you would really like to get to the root of the issue the problem with the previous posts was here...(The properties of the jobDataMap are serializable as also correctly pointed by the link you referred to.)

			<bean class="org.quartz.JobDataMap">
				<constructor-arg>
					<map>
						<entry key="jobName" value="myJob" />
						<entry key="jobLocator" value-ref="jobRegistry" />
						<entry key="jobLauncher" value-ref="jobLauncher" />
					</map>
				</constructor-arg>
			</bean>

if you need access to these jobRegistry and jobLauncher in your jobBean(QuartzJobBean as the below posts are using) all you have to do is

<bean class="org.springframework.scheduling.quartz.Sched ulerFactoryBean">
<property name="schedulerContextAsMap">
<map>
					<entry key="jobLocator" value-ref="jobRegistry" />
						<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
.......
</property>
	
</bean>

I point u to the documentation too where this problem is specifically addressed. Its almost always an urgent need that pushes programmers to put these non serializables into the jobDataMap for a quick solution.

	/**
	 * Register objects in the Scheduler context via a given Map.
	 * These objects will be available to any Job that runs in this Scheduler.
	 * <p>Note: When using persistent Jobs whose JobDetail will be kept in the
	 * database, do not put Spring-managed beans or an ApplicationContext
	 * reference into the JobDataMap but rather into the SchedulerContext.
	 * @param schedulerContextAsMap Map with String keys and any objects as
	 * values (for example Spring-managed beans)
	 * @see JobDetailBean#setJobDataAsMap
	 */
	public void setSchedulerContextAsMap(Map schedulerContextAsMap) {
		this.schedulerContextMap = schedulerContextAsMap;
	}

Ive implemented this solution and is quite elegant than having a whole application context available inside ur job from which you are getting all your beans.
I disagree to agree that this has been an existing problem with the api since it has been a mis interpretation of the responsibility of each class that has caused this confusion often because the documentation is found wanting in certain areas.
since you are alreay extending the quartzJobBean most of the times its only necessary to use the contextMap as above. Unless I missed your point