Problem:
--------
Environment has multiple threads, each running a Spring Integration context which polls a directory for a file and, once found, executes a Spring Batch job using
the Spring Batch Integration functionality (described here:http://blog.springsource.org/2010/02...g-integration/ See: Message Trigger section).

The problem was that every so often, I'd encounter a deadlock scenario where 2 threads would attempt to kick off a Spring Batch job simultaneously, resulting in a
deadlock exception when dealing with the persisted jobRepository.

Exception in log files
----------------------
[CODE]2012-09-18 12:39:18,404 ERROR [task-scheduler-1] handler.LoggingHandler (LoggingHandler.java:126) - org.springframework.integration.MessageHandlingExc eption: org.springframework.dao.DataAccessResourceFailureE xception: Could not increment identity; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 172) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at org.springframework.integration.handler.MethodInvo kingMessageProcessor.processMessage(MethodInvoking MessageProcessor.java:76)
at org.springframework.integration.handler.ServiceAct ivatingHandler.handleRequestMessage(ServiceActivat ingHandler.java:64)
at org.springframework.integration.handler.AbstractRe plyProducingMessageHandler.handleMessageInternal(A bstractReplyProducingMessageHandler.java:97)
at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:73)_
at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :114)
at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:1 01)
at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:61)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:128)
at org.springframework.integration.core.MessagingTemp late.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemp late.send(MessagingTemplate.java:149)
....
at java.lang.Thread.run(Thread.java:662)
Caused by: org.springframework.dao.DataAccessResourceFailureE xception: Could not increment identity; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 172) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at org.springframework.jdbc.support.incrementer.SqlSe rverMaxValueIncrementer.getNextKey(SqlServerMaxVal ueIncrementer.java:108)
at org.springframework.jdbc.support.incrementer.Abstr actDataFieldMaxValueIncrementer.nextLongValue(Abst ractDataFieldMaxValueIncrementer.java:125)
at org.springframework.batch.core.repository.dao.Jdbc JobInstanceDao.createJobInstance(JdbcJobInstanceDa o.java:110)
at org.springframework.batch.core.repository.support. SimpleJobRepository.createJobExecution(SimpleJobRe pository.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:183)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :150)
at org.springframework.transaction.interceptor.Transa ctionInterceptor.invoke(TransactionInterceptor.jav a:110)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :172)
at org.springframework.batch.core.repository.support. AbstractJobRepositoryFactoryBean$1.invoke(Abstract JobRepositoryFactoryBean.java:168)
at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :172)
at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy12.createJobExecution(Unknown Source)
at org.springframework.batch.core.launch.support.Simp leJobLauncher.run(SimpleJobLauncher.java:111)
at org.springframework.batch.integration.launch.JobLa unchingMessageHandler.launch(JobLaunchingMessageHa ndler.java:49)
...


Initially, I tried configuring some retriable exceptions in my Spring Batch configuration and was still receiving the exceptions. I noticed the exception was actually thrown **before** my
Batch functionality was invoked, which is why retriable exceptions wasn't working for me.

Solution:
--------
Use AOP to use a RetryTemplate on the Batch Integration job launcher!! The existing Spring Batch documentation mentions how to do this, but I wanted to provide an example of how I got it to work using
Batch Integration's JobLauncher.

Basically, the Spring Batch Integration JobLaunchingMessageHandler invokes the Batch SimpleJobLauncher. I configured AOP to basically "listen" for exceptions on my SimpleJobLauncher and
retry the job launch if they occur (for a max: 7 times).

This setup will also log scenarios when this deadlock occurs!!! So far, the following configuration has worked for me flawlessly and now retries deadlock exceptions.
Code:
    <!--Begin Batch/AOP retry config.  This is used to retry on potential deadlocks on SimpleJobRepository.createJobExecution()-->
    <bean id="batchRetryPolicy" class="org.springframework.batch.retry.policy.SimpleRetryPolicy" >
        <property name="maxAttempts" value="7"></property>
        <property name="retryableExceptions">
            <util:map>
                <entry key="org.springframework.dao.DeadlockLoserDataAccessException" value="true"/>
                <entry key="org.springframework.dao.DataAccessResourceFailureException" value="true"/>
            </util:map>
        </property>
    </bean>

    <bean id="batchRetryTemplate" class="org.springframework.batch.retry.support.RetryTemplate" >
        <property name="retryPolicy" ref="batchRetryPolicy"></property>
        <property name="listeners">
            <bean id="retryLogger" class="com.wf.fx.tradefeed.batch.aop.LoggingRetryListener"/>
        </property>
    </bean>

    <bean id="batchRetryAdvice"
          class="org.springframework.batch.retry.interceptor.RetryOperationsInterceptor" >
        <property name="retryOperations" ref="batchRetryTemplate"></property>
    </bean>

    <aop:config>
        <aop:pointcut id="launching"
                      expression="execution(* org.springframework.batch.core.launch.support.SimpleJobLauncher.*(..))"></aop:pointcut>
        <aop:advisor pointcut-ref="launching" advice-ref="batchRetryAdvice" order="-1"></aop:advisor>
    </aop:config>
    <!--End Batch/AOP retry config-->
I wanted to share my solution in case anyone else encountered this issue. Good luck!!!

-Chris