FlatFileItemWriter: Underlying CharsetEncoder not configurable

[Fiouz]

Hello,

While trying to write unsupported characters to a file (using the FlatFileItemWriter "encoding" property), I'm getting an UnmappableCharacterException:

ERROR [main] org.springframework.transaction.support.TransactionSynchronizationUtils | TransactionSynchronization.afterCompletion threw exception
org.springframework.batch.support.transaction.FlushFailedException: Could not write to output buffer
        at org.springframework.batch.support.transaction.TransactionAwareBufferedWriter$1.afterCompletion(TransactionAwareBufferedWriter.java:71)
        at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:157)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:974)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:949)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:777)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
        at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:304)
        at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
        at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
        at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
        at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
        at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:242)
        at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198)
        at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:348)
        at org.springframework.batch.core.job.flow.FlowJob.access$0(FlowJob.java:1)
        at org.springframework.batch.core.job.flow.FlowJob$JobFlowExecutor.executeStep(FlowJob.java:135)
        at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
        at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
        at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:103)
        at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:250)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:110)
        at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
        at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:105)
        at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:291)
        at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:448)
        at xxx
Caused by: java.nio.charset.UnmappableCharacterException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:261)
        at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(StreamEncoder.java:398)
        at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:136)
        at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:146)
        at java.io.Writer.write(Writer.java:126)
        at org.springframework.batch.support.transaction.TransactionAwareBufferedWriter$1.afterCompletion(TransactionAwareBufferedWriter.java:67)
        ... 26 more

So far, that is expected since my target encoding has no match for my data.
The problem is that I can't tell the writer not to fail when such event occurs.

Since FlatFileItemWriter (2.0.3) uses java.nio.channels.Channels.newWriter(WritableByteC hannel, String) instead of java.nio.channels.Channels.newWriter(WritableByteC hannel, CharsetEncoder, int) (in a private inner class), I can't find an easy way to configure the underlying CharsetEncoder in order to set the unmappableCharacterAction without duplicating lots of code.

Is there an easy way to achieve this?

Thanks

[Dave Syer]

I think you are supposed to use the CharsetProvider SPI to register your Charset (and encoder). It seems like an unmappable character is an error any way you look at it, so you probably need to track down or make a Charset implementation for your stream rather than trying to ignore the exception?

[Fiouz]

Thanks,

You're right, such errors should be reported and not simply ignored, and implementing a Charset/CharsetEncoder seems to be a way to track them down without throwing an exception.

I'll look into implementing my own ISO-8859-1 delegating charset (I just need to alter the way unmappable characters are handled), if possible.