PagedResultsRequestControl and random exceptions

[jmbaker]

Hi,

I'm looking PagedResultsRequestControl and I'm unsure if it works correctly with LdapTemplate. Here's my bean definition:

<bean id="global.ldapTemplate" class="org.springframework.ldap.core.LdapTemplate" >
<constructor-arg ref="datasource.ldap" />
</bean>
<bean id="datasource.ldap" class="org.springframework.ldap.core.support.LdapC ontextSource">
<property name="url" value="ldap://intranet:389" />
<property name="base" value="xxx" />
<property name="userName" value="xxx" />
<property name="password" value="xxx" />
</bean>

And my code:

List<User> users = new ArrayList<User>();
final int PAGE_SIZE = 100;
PagedResultsRequestControl control = new PagedResultsRequestControl(PAGE_SIZE, null);
do
{
SearchControls searchControls = new SearchControls();
searchControls.setSearchScope(SearchControls.SUBTR EE_SCOPE);
List<User> temp = (List<User>)ldapTemplate.search(baseDn,
memberOf.encode(), searchControls,
new UserAttributesMapper(), control);
users.addAll(temp);
logger.info("blip");
control = new PagedResultsRequestControl(PAGE_SIZE, control.getCookie());
} while (control.getCookie().getCookie() != null);

However when I run the application I keep getting the following exception:

org.springframework.ldap.OperationNotSupportedExce ption: [LDAP: error code 12 -00000057: LdapErr: DSID-0C09068F, comment: Error processing control, data 0, vece...
at org.springframework.ldap.support.LdapUtils.convert LdapException(LdapUtils.java:171)
at org.springframework.ldap.core.LdapTemplate.search( LdapTemplate.java:295)
at org.springframework.ldap.core.LdapTemplate.search( LdapTemplate.java:234)
at org.springframework.ldap.core.LdapTemplate.search( LdapTemplate.java:548)

The problem with this bug is that it's random - you'll note the logger.info("blip") so I can monitor the iterations of the loop. The number of blips output is entirely random! I've read a few posts on this subject and some have suggested connection pooling may be a problem, and hence I'm wondering if LdapTemplate can ever do the job properly given it closes the context after the search has been executed.

Does anyone have a concrete example of this functionality working for thousands of records in the LDAP?

Thanks,


John

[rasky]

The problem is most likely related to the closed context. In the upcoming release (1.3-rc1 is due in a couple of days) we provide a SingleContextSource implementation, which will make sure that the actual target connection is never closed.

[jmbaker]

Hello,

I did try and override the search method in LdapTemplate and comment out the calls to close the context but this still failed. Can you tell me where to find 1.3rc1 (or some recent build) and I'll happily try out your suggestion? Perhaps you can also post some example config for this new context source?

Thanks,


John

[jmbaker]

Rasky,

On further consideration, do we not need a search mechanism that specifically does not call context close? The lifetime of the connection needs to be guaranteed over the multiple calls to ldapTemplate.search.


John

[rasky]

The explicit call to close is imperative in LdapTemplate to prevent connection leaks - there's really no way around that.

What the SingleContextSource does is that it creates a proxy around the target DirContext instance which ignores any calls to close, so in effect close will actually never be called on the target DirContext.

You can accomplish the same behavior in 1.2.1 using the built-in client-side transaction support: declare a ContextSourceTransactionManager, create TransactionAwareContextSourceProxy around your ContextSource, and then perform your loop within a transaction boundary. That will make sure the same actual target connection (DirContext) will be used throughout the entire transaction. Check out the reference docs for more information on how to configure this.

If you want to try out the SingleContextSource you can get the latest code from svn trunk:

svn co https://springframework.svn.sourceforge.net/svnroot/springframework/spring-ldap/trunk spring-ldap
cd spring-ldap
mvn install

[jmbaker]

Hi,

I've implemented the transaction manager but it still fails - in fact, the LDAP query fails on the second attrempt, without fail. This is actually worse than not using the transaction manager as it did at least do a few iterations before failing! Here's the exception:

Caused by: javax.naming.CommunicationException: [LDAP: error code 2 - 00000057: LdapErr: DSID-0C09068F, comment: Error processing control, data 0, vece
	at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:98)
	at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:295)
	at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:234)
	at org.springframework.ldap.core.LdapTemplate.search(LdapTemplate.java:548)

I've debugged the transaction manager and I can see TransactionAwareDirContextInvocationHandler being invoked, catching the close method, calling doCloseTransaction and calling this:

        } else {
            log.debug("Leaving transactional context open");
        }

So it would appear I've set it up correctly. However here is the Spring configuration:

  <bean id="contextSource" class="org.springframework.ldap.transaction.compensating.manager.TransactionAwareContextSourceProxy">
    <constructor-arg ref="datasource.ldap" />
  </bean>        
  <bean id="transactionManager" 
        class="org.springframework.ldap.transaction.compensating.manager.ContextSourceTransactionManager">
    <property name="contextSource" ref="contextSource"/>
  </bean>    
    
  <!-- LDAP template to use with the LDAP datasource -->
  <bean id="global.ldapTemplate" class="org.springframework.ldap.core.LdapTemplate">
    <constructor-arg ref="contextSource" />
  </bean>   
    
  <!-- User management DAO -->
  <bean id="global.ldap.xx.userManagementDAO.target" 
    class="com.xx.dao.ldap.UserManagementDAO">
    <property name="ldapTemplate" ref="global.ldapTemplate"/>
    <!-- BaseDN is RELATIVE to datasource base -->
    <property name="baseDn" value="ou=GLB" />
  </bean>      
  
  <bean id="global.ldap.xx.userManagementDAO" 
          class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">
    <property name="transactionManager" ref="transactionManager" />
    <property name="target" ref="global.ldap.xx.userManagementDAO.target" />
    <property name="transactionAttributes">
       <props>
          <prop key="*">PROPAGATION_REQUIRES_NEW</prop>
       </props>
    </property>
  </bean>

Do you have any further thoughts? Is it possible our LDAP doesn't support multiple queries in one connection?


John

[jmbaker]

Hi,

This works correctly:

      try 
      {
         // Create the initial directory context
         LdapContext ctx = (LdapContext)ldapTemplate.getContextSource().getReadOnlyContext();

         // Create the search controls
         SearchControls searchControls = new SearchControls();

         // Specify the attributes to return
         String returnedAtts[] = {"cn", "sAMAccountName", "mail"};
         searchControls.setReturningAttributes(returnedAtts);

         // Specify the search scope
         searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);

         // Set the page size and initialize the cookie that we pass back in subsequent pages
         byte[] cookie = null;

         // Request the paged results control
         Control[] ctls = new Control[]{new PagedResultsControl(PAGE_SIZE, true)};
         ctx.setRequestControls(ctls);
         
         // Search for objects using the filter
         do 
         {
            NamingEnumeration results = ctx.search(base, memberOf.encode(), searchControls);
            while (results != null && results.hasMoreElements()) {
               SearchResult sr = (SearchResult) results.next();
               users.add((User)mapper.mapFromAttributes(sr.getAttributes()));
            }
            
            // examine the response controls
            cookie = parseControls(ctx.getResponseControls());

            // pass the cookie back to the server for the next page
            ctx.setRequestControls(new Control[]{
                  new PagedResultsControl(PAGE_SIZE, cookie, Control.CRITICAL)});

         } while ((cookie != null) && (cookie.length != 0));

         ctx.close();
     }
     catch (NamingException e) 
     { logger.warn(e.getMessage(), e); }
     catch (IOException e) 
     { logger.warn(e.getMessage(), e); }

So I can only assume there's a problem with the paging code?


John

[jmbaker]

Hi,

Any further thoughts on this unresolved issue?


John

[rasky]

I honestly don't know what's going wrong here. LDAP error 12 means "Unavailable Critical Extension Requested". I have so far been unable to reproduce. Could you debug and set a breakpoint in AbstractRequestControlDirContextProcessor#preProce ss(). It might be interesting to see the contents of the newControls array at the end the second time you perform the search.

[whiteroses009@yahoo.com]

Hi,
Did you figure out this problem ?