handleLine exception handling for skippable header

[dave_m_burns]

Hi,
First up, great work on Spring batch, I've been using it for about 5 weeks and it does a great job for what we need, the forums are also great for getting tips on the best implementations.

I've a quick query around handling of a header row with a flatFileItemReader. I have a an input file with the following format :

DEVENV|DEV|23569|2011x-01-12-03-02-45
sss|1234|555|6666
sss|1234|555|6667
sss|1234|555|6668
sss|1234|555|6669


which is basically a header with some strings to identify the batch load file and a timestamp, followed by the actual item rows. In the above example I've created a bad format date and wanted the batch to stop in this test scenario, i.e. don't process any of the rows.

Based on the following config and code everything nearly works to plan. The problem is I'm using spring batch admin and I want only the unexpected job exec excecution exception to come out as the exit message in the GUI, where as the stackl trace is trunc'ed on output and I get only the top level ItemStreamException in the output exit message :

11:51:21,530 ERROR jobLauncherTaskExecutor-1 step.AbstractStep:212 - Encountered an error executing the step
org.springframework.batch.item.ItemStreamException : Failed to initialize the reader
at org.springframework.batch.item.support.AbstractIte mCountingItemStreamItemReader.open(AbstractItemCou ntingItemStreamItemReader.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
...

My expilict ""Failed to Parse header file creation Date ..." message is trunc'ed and so not seen in the output table.

Just wondering if I can force my error into the exit message using the handlLine callback otherwise I suppose I could write an explicit tasklet to do this functionality, just thought it was cleaner trying the above approach.

<step id="flatFileToDatabase" parent="simpleStep" >
			<tasklet>
				<chunk reader="batchFileItemReader" writer="batchItemWriter" commit-interval="${job.commit.interval}" />
				<listeners>
            		<listener ref="headerCopier" />
       			</listeners>
			</tasklet>
			<fail on="FAILED"/>
    		<next on="*" to="transformToPDS"/>  
		</step>

<bean id="batchFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
				<property name="resource">
			<bean class="org.springframework.core.io.FileSystemResource">
				<constructor-arg value="#{jobExecutionContext['BATCH_LOAD_FILE']}"/> 
			</bean>
		</property>
		<property name="lineMapper">
			<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
				<property name="lineTokenizer">
					<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
						<property name="names" value="data" />
					</bean>
				</property>
				<property name="fieldSetMapper">
					<bean class="ie.bil.projections.batch.mapper.BatchFileLoadItemMapper" />
				</property>
			</bean>
		</property>
		<property name="skippedLinesCallback" ref="headerCopier"/>
		<property name="linesToSkip" value="1" />
	</bean>

<bean id="headerCopier" class="ie.bil.projections.batch.support.HeaderCopyCallback">
	 	<property name="batchTableHeaderDao">
			<bean class="ie.bil.projections.batch.dao.BatchTableHeaderDaoImpl">
				<property name="dataSource" ref="dataSource" />
			</bean>
		</property>
	</bean>

public class HeaderCopyCallback extends StepExecutionListenerSupport 
					implements LineCallbackHandler, FlatFileHeaderCallback {

	private BatchTableHeaderDao batchTableHeaderDao_c;
	private EmailService emailService_c;

	private StepExecution stepExecution_c;
	
	private static Logger logger_c = Logger.getLogger(HeaderCopyCallback.class);
	
	// Header Field Locations
	private static int SYSTEM = 0;
	private static int LEVEL_REF = 1;
	private static int SCHEDULE_NUMBER = 2;
	private static int FILE_CREATED_DATE = 3;
	
	private static String DATE_PATTERN = "yyyy-dd-MM-HH-mm-ss";
	
	/**
	 * Stores the details for the batch based on the first/header row in the
	 * batch file. These details are used to audit the batch progress 
	 * throughout the remaining batch steps.
	 * 
	 * @param line_p	The header line/row within the batch file
	 */
	public void handleLine(String line_p)  {
		Assert.notNull(line_p);
		
		/*
		 * Here we can't use a field set mapper to extract/parse the 
		 * individual string elements so manually doing the parse for now.
		 */
		String [] headerFields = StringUtils.tokenizeToStringArray(line_p, "|");
		
		BatchTableHeader batchHeader = new BatchTableHeader(); 
		batchHeader.setSystem(headerFields[SYSTEM]);
		batchHeader.setLevelRef(headerFields[LEVEL_REF]);
		batchHeader.setScheduleNumber(headerFields[SCHEDULE_NUMBER]);
		
		// If we can't parse the date then fail the whole job.
		SimpleDateFormat format = new SimpleDateFormat(DATE_PATTERN);
		Date fileCreationDate = null;
		try {
			fileCreationDate = format.parse(headerFields[FILE_CREATED_DATE]);
		} catch (ParseException pe) {
		
			String errMsg = "Failed to Parse header file creation Date : " + headerFields[FILE_CREATED_DATE];
			logger_c.error(errMsg);
			
			throw new UnexpectedJobExecutionException(errMsg);
		}

		batchHeader.setFileCreationTimestamp(fileCreationDate);
		batchHeader.setJobExecutionId(stepExecution_c.getJobExecutionId());
		
		batchTableHeaderDao_c.save(batchHeader);
	}

[Dave Syer]

I suppose you could get the full stack trace by changing the DB schema and setting max-varchar-length sufficiently high in the JobRepository. That's the most straightforward thing you could do. Otherwise you could add job or step execution listener to pull out the relevent information and replace it in the ExitStatus, instead of appending. Probably I would prefer to the latter option. A StepExecutionListener should have access to the volatile execution exceptions kept in the StepExecution.