Page 1 of 3 123 LastLast
Results 1 to 10 of 23

Thread: Configuring a job for online system

  1. #1
    Join Date
    Nov 2006
    Posts
    9

    Default Configuring a job for online system

    I Need help figuring out how to setup a batch to use existing online objects, specifically my service layer and corresponding dao's. I create a job that ran completely external to the online application as a demo However I duplicated a lot of objects and logic. Now I would like to use my service layer, in my ItemWriter, so I can reuse the online logic instead of duplicating. The online system uses Spring and Hibernate and the DAOs and service objects use spring DI to get refernce to the sessionFactory, datasource configured in the application context. I would like to run everything from the online system (manual jobs and scheduled jobs). Right now I would like the controller to be able to launch a job ( synchronously or asynchronously). What I tried was to used imports in the psring batch config for the application context file and I tried ot kick it off like I previously did with the CommandLineJobRunner and it went into an infinit loop. I am not sure how this needs to be setup and launched, can someone give me a rundown of how to kick off the job from a servlet and how to configure the batch files so the exisitng DAOs get access to the sessionFactory and datasource? I don't care about running synchronously or asynchronously at this point.

    Thank you,
    Todd

  2. #2
    Join Date
    Nov 2006
    Posts
    9

    Default

    Okay, I got the job to run and figured out that I had a cyclic reference in the config (xml) files imports. I added a new configuration for the sessionFactory in the launcher xml and referenced the service object and DAOs by putting their config files on the classpath and imported them in my launcher xml. Then I kicked off the job with the command line runner...

    CommandLineJobRunner.main(new String[] {"InventoryJob.xml", "inventoryDailyJob", "schedule.date(string)=" + df.format(new Date())});

    The above turned out to have a few issue, the appserver was shut down after each run and when running on tomcat I kept getting a random null pointer exception that looked like it was loosing connections?? I ended up kicking off the job with the job launcher I had configured in my launch xml like so..

    JobParameters jobParameters = new JobParametersBuilder().addDate ("schedule.date", new Date());
    JobExecution run = jobLauncher.run(job, jobParameters);

    I also added the SimpleAsyncTaskExecutor to the job launcher and now the servlet is not waiting on the job. I think I got this resolved.

  3. #3
    Join Date
    Jan 2008
    Location
    San Diego
    Posts
    780

    Default

    Quote Originally Posted by tbone21w View Post
    Okay, I got the job to run and figured out that I had a cyclic reference in the config (xml) files imports. I added a new configuration for the sessionFactory in the launcher xml and referenced the service object and DAOs by putting their config files on the classpath and imported them in my launcher xml. Then I kicked off the job with the command line runner...

    CommandLineJobRunner.main(new String[] {"InventoryJob.xml", "inventoryDailyJob", "schedule.date(string)=" + df.format(new Date())});

    The above turned out to have a few issue, the appserver was shut down after each run and when running on tomcat I kept getting a random null pointer exception that looked like it was loosing connections?? I ended up kicking off the job with the job launcher I had configured in my launch xml like so..

    JobParameters jobParameters = new JobParametersBuilder().addDate ("schedule.date", new Date());
    JobExecution run = jobLauncher.run(job, jobParameters);

    I also added the SimpleAsyncTaskExecutor to the job launcher and now the servlet is not waiting on the job. I think I got this resolved.
    You don't want to use a commandline runner to launch a batch job from within your web application.

    I'm using the following:

    Code:
    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    		<property name="jobRepository" ref="jobRepository" />
    		<property name="taskExecutor">
    		  <bean class="org.springframework.core.task.SimpleAsyncTaskExecutor" />
    		</property>
    	</bean>
    
    	<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
    	  <property name="databaseType" value="sqlserver"/>
    	  <property name="dataSource" ref="dataSource"/>
    	  <property name="transactionManager" ref="transactionManager"/>
    	</bean>
    Then I launch the job from within a helper class called by my controller:

    Code:
                Job batchJob = jobFactory.createJob();
                // You would want to add any parameters for your job here
                JobParameters parameters = new JobParameters(map, new HashMap<String,Long>(), new HashMap<String,Double>(), new HashMap<String,Date>());
               
                // Now, kick off the job
                jobLauncher.run(batchJob, parameters);
    Note that many of the beans that you will interact with in a batch job are stateful so you cannot define them as spring singleton beans. What I did was wire up all of my job beans and their associated readers/writers in a separate spring bean config that is not loaded into my app context at startup. Then, I use this implementation for my job factory so that everytime that I create a job to run, it is created in it's own sub context using the parent app context of the web app and the prototype beans in the separate bean config:

    Code:
    public class ContextAwareJobFactory implements JobFactory, ApplicationContextAware, InitializingBean
    {
        private ClassPathXmlApplicationContextJobFactory delegate;
        
        /* The parent application context */
        private ApplicationContext applicationContext;
        /* The job bean name */
        private String beanName;
        /* resource path to subcontext spring config */
        private String subcontextPath;
    
        public void setBeanName(String beanName)
        {
            this.beanName = beanName;
        }
    
        public void setSubcontextPath(String subcontextPath)
        {
            this.subcontextPath = subcontextPath;
        }
    
        /* (non-Javadoc)
         * @see org.springframework.batch.core.configuration.JobFactory#createJob()
         */
        @Override
        public Job createJob()
        {
            return delegate.createJob();
        }
    
        /* (non-Javadoc)
         * @see org.springframework.batch.core.configuration.JobFactory#getJobName()
         */
        @Override
        public String getJobName()
        {
            return delegate.getJobName();
        }
    
        /* (non-Javadoc)
         * @see org.springframework.context.ApplicationContextAware#setApplicationContext(org.springframework.context.ApplicationContext)
         */
        @Override
        public void setApplicationContext(ApplicationContext context) throws BeansException
        {
            this.applicationContext = context;
        }
    
        /* (non-Javadoc)
         * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet()
         */
        @Override
        public void afterPropertiesSet() throws Exception
        {
            delegate = new ClassPathXmlApplicationContextJobFactory(this.beanName, this.subcontextPath, this.applicationContext);
        }
    }
    The job factory is declared as such:

    Code:
    <bean id="jobFactory" class="com.foobar.ContextAwareJobFactory">
          <property name="beanName" value="provisioningBatchUploadJob"/>
          <property name="subcontextPath" value="classpath:spring/fubar-project-batch-processing-prototype-beans.xml"/>
        </bean>
    The code in my helper class above is invoked directly from within my spring mvc controller. In my case, I'm taking an uploaded file from the user and using that as the source of the batch input. After futzing around with it for a few days, I finally got all the issues sorted out and now it is working like a champ. I've even verified that the restart behavior works correctly, allowing an admin to restart a failed batch by fixing the input file and restarting from the web interface.

  4. #4
    Join Date
    Nov 2006
    Posts
    9

    Default

    chudak thank you for the configuration, this is exactly what I have been trying to do. I moved all my job beans to a new config file and declared them as prototype. I also created the ContextAwareJobFactory and wired it up in the application context. In my helper class that kicks of the job I have access to the JobFactory since it is part of the application context (and I have it configured/wired in the application context). However, I am having trouble getting reference to my job launcher that is configured in the job bean config file. How did you wire up the helper class to the launcher?

    Thank you for the help,
    Todd
    Last edited by tbone21w; Sep 28th, 2008 at 09:32 AM.

  5. #5
    Join Date
    Jan 2008
    Location
    San Diego
    Posts
    780

    Default

    Quote Originally Posted by tbone21w View Post
    chudak thank you for the configuration, this is exactly what I have been trying to do. I moved all my job beans to a new config file and declared them as prototype. I also created the ContextAwareJobFactory and wired it up in the application context. In my helper class that kicks of the job I have access to the JobFactory since it is part of the application context (and I have it configured/wired in the application context). However, I am having trouble getting reference to my job launcher that is configured in the job bean config file. How did you wire up the helper class to the launcher?

    Thank you for the help,
    Todd
    You don't need to declare the beans as prototypes if you move them to a separate bean file and use the job factory as I've indicated (in fact it can cause problems). The 'prototype'ness is provided by the job factory.

    The job launcher is not stateful so it should be in your parent app context. Only stateful beans (or beans that are injected with stateful beans) should be in the sub context bean file.

  6. #6
    Join Date
    Nov 2006
    Posts
    9

    Default

    I moved the JobLauncher and JobRepository to the application context and made sure the sub context beans were not declared as prototype and everything seams to be working. I was using an earlier version of Spring Batch and moved to 1.1.2 version. In doing so I had to change import references on a coulple of classes and I had to rebuild the batch tables based on the new schema. Thanks for all the help.

  7. #7
    Join Date
    Sep 2008
    Posts
    20

    Default

    Quote Originally Posted by chudak View Post

    Code:
    <bean id="jobFactory" class="com.foobar.ContextAwareJobFactory">
          <property name="beanName" value="provisioningBatchUploadJob"/>
          <property name="subcontextPath" value="classpath:spring/fubar-project-batch-processing-prototype-beans.xml"/>
        </bean>
    .
    Chudak, Thanks for posting this example. Even i'm thinking of using this code in my application. But i've several jobs in my web applications and it gets invoked from the user through User Interface. So, please suggest how i can avoid hardcoding JOBNAME in the above configuration

  8. #8
    Join Date
    Jan 2008
    Location
    San Diego
    Posts
    780

    Default

    Quote Originally Posted by Jonathan_r View Post
    Chudak, Thanks for posting this example. Even i'm thinking of using this code in my application. But i've several jobs in my web applications and it gets invoked from the user through User Interface. So, please suggest how i can avoid hardcoding JOBNAME in the above configuration
    In the new spring batch, there is a class ClassPathXmlJobRegistry that ostensibly does what the above does but allows you to register multiple jobs. You still have to name them (the name refers to a configured job spring bean).

  9. #9

    Default

    Chudak,

    Thanks for this excellent posting.

    Can you please explain in your framework how you are capturing the errors which is happening during file processing and providing that errors back to the user in the UI.

    Your help on this is much appreciated.

    Thanks,
    vbforums

  10. #10
    Join Date
    Jan 2008
    Location
    San Diego
    Posts
    780

    Default

    Quote Originally Posted by vbforums View Post
    Chudak,

    Thanks for this excellent posting.

    Can you please explain in your framework how you are capturing the errors which is happening during file processing and providing that errors back to the user in the UI.

    Your help on this is much appreciated.

    Thanks,
    vbforums
    I'm not capturing errors and reporting them to the user in the UI.

    (My) Batch processing is asynchronous and, in the case of tens of thousands of records, can take upwards of a half hour to process. I only return an error to the browser user if there is a synchronous error/failure in STARTING the job processing.

    What I AM doing is configuring a job execution listener that sends out an email in case of failure to a mailing list so that an administrator can take the appropriate action to fix and restart the batch job.

    Code:
    public class ProvisioningBatchJobErrorExecutionListener extends JobExecutionListenerSupport
    {
        private MailSender mailSender;
        private String[] recipients;
        private String sender;
        private String subject;
        private String message;
        
        /**
         * Set the mail sender.
         * 
         * @param mailSender the sender
         */
        public void setMailSender(MailSender mailSender)
        {
            this.mailSender = mailSender;
        }
    
        /**
         * Set the recipients for the message.
         * 
         * @param recipients the recipients
         */
        public void setRecipients(String[] recipients)
        {
            this.recipients = recipients;
        }
    
        /**
         * Set the sender for the email.
         * 
         * @param sender the sender
         */
        public void setSender(String sender)
        {
            this.sender = sender;
        }
    
        /**
         * Set the subject to be sent for the email.
         * 
         * @param subject the subject string
         */
        public void setSubject(String subject)
        {
            this.subject = subject;
        }
    
        /**
         * Set the message format style message to be sent
         * in the email body. The batch guid, batch filename and
         * stacktrace (as a string) will be used as arguments, in that order.
         * 
         * @param message the message format style string
         */
        public void setMessage(String message)
        {
            this.message = message;
        }
    
        @Override
        public void onError(JobExecution jobexecution, Throwable throwable)
        {
            String batchGuid = jobexecution.getJobInstance().getJobParameters().getString(Constants.GUID);
            String batchFile = jobexecution.getJobInstance().getJobParameters().getString(Constants.UPLOAD_FILENAME);
            
            String formattedMessage = MessageFormat.format(this.message, batchGuid, batchFile, ExceptionUtils.getStackTrace(throwable));
            
            SimpleMailMessage mailMessage = new SimpleMailMessage();
            mailMessage.setFrom(this.sender);
            mailMessage.setTo(this.recipients);
            mailMessage.setSubject(this.subject);
            mailMessage.setText(formattedMessage);
            
            mailSender.send(mailMessage);
        }
    }
    Configured as a bean:

    Code:
    <bean id="provisioningBatchErrorListener" 
              class="com.(elided)provisioning.batch.ProvisioningBatchJobErrorExecutionListener">
          <property name="mailSender" ref="mailSender"/>
          <property name="sender" ref="mailReplyTo"/>
          <property name="recipients" ref="mailRecipients"/>
          <property name="subject" value="${provisioning.inbound.edf.notification.host}: Provisioning Batch Upload Processing Error"/>
          <property name="message">
              <value>
    Batch Guid: {0}
    Batch File: {1}
    Host: ${provisioning.inbound.edf.notification.host}
    
    The processing of this file has failed. Please fix the error and restart the batch using the above guid.
    
    Details:
    
    {2}
              </value>
          </property>
        </bean>
    Then this is registered with the job:
    Code:
        <!-- Prototype job bean -->
        <bean id="simpleJob" class="org.springframework.batch.core.job.SimpleJob" abstract="true">
    		<property name="jobRepository" ref="jobRepository" />
    		<property name="restartable" value="true" />
    		<property name="jobExecutionListeners" ref="provisioningBatchErrorListener"/>
    	</bean>

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •