Duplicate match with distance <= 5 found for this property in input keys

[javed]

Hi All,
am currently getting the following exception:

Caused by: com.project.batch.exception.AppException: Invalid property 'vatStatus' of bean class [com..batch.value.input.InvoiceAll]: Duplicate match with distance <= 5 found for this property in input keys: [vatStatus, vatExemptionReason, invoiceTotalAmountSign, currency, vatNumber, countryInvoice, extractDate, endInvAllFiller, legacyInvoiceKey, extractTime, systemId, invoiceTotal, invoiceNumber, invoiceIssueDate, recordType]. (Consider reducing the distance limit or changing the input key names to get a closer match.)   LINE NUMBER: 7   LINE: INV-ALL2011012814281314HHRU 20000025535404185677      VAT          201101240000006896908 RUBR

Am currently using a taskexcutor in my codes with a value set to 5 processing a flat file with 22000 lines . Am wondering if this could be a threading issue ?

The exception seems to be coming from class org.springframework.batch.item.file.mapping.BeanWr apperFieldSetMapper at the following line:

if (name != null) {
				if (matches.containsValue(name)) {
					throw new NotWritablePropertyException(
							cls,
							name,
							"Duplicate match with distance <= "
									+ distanceLimit
									+ " found for this property in input keys: "
									+ keys
									+ ". (Consider reducing the distance limit or changing the input key names to get a closer match.)");
				}
				matches.put(key, name);
				switchPropertyNames(properties, key, name);
			}
		}

I already have a setter for vatStatus on the object that I'd like to map the line to.

A fixed length tokenizer is also being used:

<bean id="invoiceAllTokenizer"
		class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
		<property name="names"
			value="systemId,recordType,extractDate,extractTime,legacyInvoiceKey,invoiceNumber,vatNumber,vatStatus,vatExemptionReason,invoiceIssueDate,invoiceTotal,invoiceTotalAmountSign,currency,countryInvoice,endInvAllFiller" />
		<property name="columns"
			value="1-2,3-9,10-17,18-25,26-41,42-49,50-65,66-68,69-78,79-86,87-99,100-100,101-103,104-106,107-120" />
	</bean>

Do you have any ideas what could be causing this ?

thank you,
javed

[bwawok]

Can we see more of your job?

My guess: The ItemReaders in spring batch are generally not thread-safe, and you are trying to have 5 threads read from the same reader.

[arno]

As bwawok said, most of the ItemReaders aren't thread-safe. If you want to use multi-threading in a chunk-oriented step, the reader, processor, and writer must be thread-safe. Database-based processors and writers are usually thread-safe, so you need to make the reader thread-safe. To make the FlatFileItemReader thread-safe, you need to synchronize the read method (in a subclass) and set the saveState flag to false. By doing so, you loose restartability. Note this is useful only if reading is really faster than processing and writing (which is the case most of the time).

If you don't want to make the FlatFileItemReader thread-safe, you can import the data in a staging table (no processing at all) in a first step and use partitioning to parallelize the processing in a second step (partitioning is usually easier with a database).

[Dave Syer]

There was also a concurrency bug in the BeanWrapperFieldSetMapper that we fixed in 2.1.something. Upgrade to 2.1.8 to make sure you get the latest.

[leogags]

Hi,

I have just hit this problem while using 2.1.8 release.

Earlier I was mapping fields by Name and everything was working fine. I wanted to use the auto mapping feature so switched to 'BeanWrapperFieldSetMapper'. Now the same job with the same input that was working fine earlier (using field by name mapping) throws the duplicate match error.

Caused by: org.springframework.beans.NotWritablePropertyExcep tion: Invalid property 'userId' of bean class [my.DetailRecord]: Duplicate match with distance <= 5 found for this property in input keys: [id, cancelReason, userId, LINE_ID, identifierType]. (Consider reducing the distance limit or changing the input key names to get a closer match.)

[leogags]

Fixed.

The problem was that one of my token keys did not have a corresponding property in the bean. Once I match the token keys with the bean properties it works fine. The error message was a bit miss leading