Serious bug in DirContextAdaptor?

[cherrydev]

I'm getting the following behavior using Active Directory (Win2k Server):

When I change an existing multi-valued attribute, for example, removing an item or adding an item, when a value already exists, I get (ATT_OR_VALUE_EXISTS) with the attribute that it tried to modify.

As near as I can tell, this is because for some reason DirContextAdaptor.collectModifications(Attribute, List) calls collectModifications(DirContext.ADD_ATTRIBUTE, etc...) before collectModifications(DirContext.REMOVE_ATTRIBUTE, etc...)

Basically, it's generating the ModificationItems in the wrong order, attempting to add the new attribute before removing the old one. When I reverse the order of the statements, it appears to work correctly.

I'm not really sure where to post bug reports for this, since it appears that there is no JIRA module set up for Spring-LDAP. Maybe someone can point me to the right place to be posting this?

[ulsa]

Until we have a JIRA space for Spring LDAP, this is the correct forum for any issues. We'll look into your problem.

[rasky]

It does seem like something is not functioning correctly, but the order of the ModificationItems really shouldn't matter - if the attribute value is already there, no ADD_ATTRIBUTE operation for that value should be generated.

Anyway, we need to look into it. A code example would be helpful in tracking down the problem.

[cherrydev]

Well, if you read the javadocs and the comments in DirContextAdaptor you will see that it says: "Otherwise, the attribute is a multi-value attribute, in which case all modifications to the original value (removals and additions) will be collected individually."

So, it seems to be following the strategy that it mentions, except that it does the ADD and REMOVE in the wrong order. The order of the ModificationItems is indeed significant, since reversing them produces the correct result. Unless you meant something else when you said the order of the modifications shouldn't matter?

I don't have any sample code that I can share at the moment, since it's embedded quite deeply into another framework at the moment. In a nutshell, though, I am mapping a POJO to the ldap template using mapToContext and a custom ContextMapper. When I map a modified POJO back to the context, it calls context.setAttributeValues to set the multi-valued attribute, and then, ldapTemplate.modifyAttributes(dn, context.getModificationItems()). When modifyAttributes is called with the ModificationItems currently produced by getModificationItems(), Active Directory complains. When the order of the ModificationItems is reversed, it works fine.

[rasky]

While the order of the ModificationItems is indeed the order in which the actual modification takes place, making it significant in the general case, I would like to argue that it really shouldn't matter in this particular case.

I'm unable to reproduce your problem (I'm running against Apache Directory Server, which shouldn't make a difference, but admittedly Active Directory has been known to behave strangely on occations).

If I understand your problem correctly your code is similar to the following test, which works just fine for me:

protected void onSetUp() throws Exception {
    DirContextAdapter adapter = new DirContextAdapter();
    adapter.setAttributeValues("objectclass", new String[] { "top",
            "person" });
    adapter.setAttributeValue("cn", "Some Person5");
    adapter.setAttributeValue("sn", "Person5");
    adapter.setAttributeValues("description", new String[ {"qwe", "123", "rty", "uio"});

    ldapTemplate.bind(PERSON5_DN, adapter, null);        
}

public void testModifyAttributes_DirContextAdapter_MultiAttributes(){
    DirContextAdapter adapter = (DirContextAdapter) ldapTemplate.lookup(PERSON5_DN);
    adapter.setAttributeValues("description", new String[]{"qwe", "123", "klytt", "kalle"});
        
    ldapTemplate.modifyAttributes(PERSON5_DN, adapter.getModificationItems());
        
    // Verify
    adapter = (DirContextAdapter) ldapTemplate.lookup(PERSON5_DN);
    String[] attributes = adapter.getStringAttributes("description");
    assertEquals(4, attributes.length);
    assertEquals("qwe", attributes[0]);
    assertEquals("123", attributes[1]);
    assertEquals("klytt", attributes[2]);
    assertEquals("kalle", attributes[3]);
}

The getModificationItems() above produces two ModificationItems:
* ADD_ATTRIBUTE with BasicAttribute("description") and the values "klytt" and "kalle".
* REMOVE_ATTRIBUTE with BasicAttribute("description") and the values "rty" and "uio".

Since no Attribute value already present is added the order of the ModificationItems shouldn't matter in this case.

Mabe it would help if you inspected the ModificationItems to see what is actually returned from getModificationItems(). If we can determine that the order does indeed matter due to some particular feature (defect?) in Active Directory we will of course change it, but I'd like to make sure we are actually chasing the correct problem.

[cherrydev]

It appears the reason why this is happening is that DistinguishedName forces all RDN keys to lowercase, while active directory always seems to return keys in uppercase. My multi-valued attribute is a DN, so what is happening is that the DN as stored in my mapped object is returning lowercase keys when 'toString()' is called on it, while the version being compared from the context when doing an update is coming through with uppercase keys. This is causing it to be confused and generating the wrong ModificationItems.

I really wish there was special handling for DNs in the DirContextAdaptor so that this wouldn't happen. I suppose for now I'll have to modify my mapToContext method to normalize the case of the RDN keys before passing them to the context. Thanks for your help with tracking this issue down.

[rasky]

Right, it seems like we're beginning to pinpoint the problem.

That said, I'm uncertain how to solve it. Even though the values in this case represent distinguished names they're really just Strings, not only to DirContextAdapter but to the LDAP server as well. Handling strings representing dns separately would require DirContextAdapter to have knowledle which really seems out of its general responsibility, even disregarding the problem of how it would get that knowledge.

The case mismatch problem should be solved however; the common case is that Attribute values in LDAP are case-insensitive, which probably means that DirContextAdapter should be case-insensitive as well. That wouldn't solve the entire problem for you however, as the syntax for DNs is rather loose when it comes to whitespace etc., e.g.:
cn=Some Person , ou=Some Company , ou=SE
represents the same DN as
cn=Some Person,ou=Some Comapny,ou=SE

Note that your LDAP server will probably allow you to add both the Strings above as values to the same Attribute.

Basically, I've no good idea of how to solve the general problem at the moment - any suggestions would be welcome.

[cherrydev]

Isn't there any portable way to extract the attribute syntax to know if it should be interpreted as a DN? I did some experimenting and found that you can successfully extract the Syntax OID of an attribute by doing something like in the following (taken from the Sun JNDI/LDAP Tutorial):

// Get the schema tree root
DirContext schema = ctx.getSchema("");

// Get the schema object for "cn"
DirContext cnSchema = (DirContext)schema.lookup("AttributeDefinition/cn");

Then just grab the "syntax" attribute of the cnSchema. DN is 1.3.6.1.4.1.1466.115.121.1.12, for example.

Do you think we should work with this in order to more intelligently deal with comparing DNs?

[rasky]

That is an interesting idea; it's not impossible that we could include something like that. Thanks for the input.

[cherrydev]

Perhaps a strategy for this would be a different version or option inside DirContextAdaptor that would look up and cache these syntax definitions the first time the attribute is retrieved. At that point, a set of standardized translators could be implemented to translate common data types (DNs, integers, booleans, UTC datetimes, etc...) directly into java objects in a perfectly consistent manner. This would also take the workload off the developer when mapping to and from the context. I'd be happy to work on this if there's interest in it.

In a related issue, I've also been working on a simple mapping system to allow you to map ldap objects to java objects by declaring the mappings in a spring-compatible manner. If there's interest in this, I could try to polish it up a little, or at least contribute what I have now to the project, assuming the client I wrote it for agrees.