Memory leak in spring batch 1.1.4 BatchSqlUpdateItemWriter

[slade]

Hi guys,

i use spring batch 1.1.4

java version "1.5.0_19"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02)
Java HotSpot(TM) Server VM (build 1.5.0_19-b02, mixed mode)

I running my JVM with -Xms10m -Xmx64m

My batch application reads text file and writes it to database using
org.springframework.batch.item.file.FlatFileItemRe ader
and
org.springframework.batch.item.database.BatchSqlUp dateItemWriter
in skipLimitStep
with commit-interval=1000.

When my input CSV file contains data which causes database constraint violations,
JVM finally throws OutOfMemory error.

In Eclipse Memory Analyser i investigated that
org.springframework.batch.item.database.AbstractTr ansactionalResourceItemWriter
accumulates sql batch update records in
private Set failed = new HashSet();
(see AbstractTransactionalResourceItemWriter.flush() method) every time when butch update fails.
But neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter never flushes this private Set failed variable.

There is not any registered issue for this bug in http://opensource.atlassian.com/proj...g/browse/BATCH

Is this bug will be fixed in spring batch 1.1.x?

Is this bug fixed in spring batch 2.0.x or 2.1.x?

Thank you.

[Dave Syer]

Neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter are part of the framework in 2.x, so I suggest you upgrade. If you need bug fixes in 1.1.4 I think we still support it commercially for about another year or so. It might not be a bug though, or you can work around it at least, by flushing in a listener (probably).

[slade]

Neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter are part of the framework in 2.x

Thank you for good news.

It might not be a bug though, or you can work around it at least, by flushing in a listener (probably).

Yes, probably "failed" variable is used for rollbacks somehow...
Unfortunately it's impossible to flush it in listener, AbstractTransactionalResourceItemWriter doesn't have public/protected method for accessing it.