LDAP Connection - MaxActive Exceeded and unexpected pooling behaviour

[kirenpillay]

Hi All,

I've been looking at this for a while now. There are 2 issues:
1. I can't get the number of idle active connections to stay idle, they seem to be closed all the time.
2. The number of connections to my LDAP server exceeds the maxActive value.

It could be that my close() calls are closing the connections and not returning the connection back to the pool, but actually closing the underlying socket?

Config:

 
 
ldap.testOnBorrow=false
ldap.testWhileIdle=false
 
ldap.maxActive=2
ldap.maxIdle=8
ldap.minIdle=4
ldap.maxWait=10000
ldap.whenExhaustedAction=0 
ldap.timeBetweenEvictionRunsMillis=300000
ldap.numTestsPerEvictionRun=3
#10 minutes before connection becomes evictable
ldap.minEvictableIdleTimeMillis=600000

Application Code:

public class CurAttributeDaoImpl implements CurAttributeDao {

	private final static Logger LOGGER = Logger
			.getLogger(CurAttributeDaoImpl.class);
	public static final String BILLING_PLATFORM_ID = "billingPlatformId";
	@Autowired
	private PoolingContextSource poolingContextSource;

	public void initialise() throws NamingException {
		DirContext ctx = poolingContextSource.getContextSource()		
				.getReadOnlyContext();
	
		
		LOGGER.info("Pool Config Values:");
		LOGGER.info("maxActive:"+poolingContextSource.getMaxActive());
		LOGGER.info("maxIdle:"+poolingContextSource.getMaxIdle());
		LOGGER.info("minIdle:"+poolingContextSource.getMinIdle());
		LOGGER.info("maxWait:"+poolingContextSource.getMaxWait());
		LOGGER.info("whenExhaustedAction:"+poolingContextSource.getWhenExhaustedAction());
		LOGGER.info("timeBetweenEvictionRunsMillis:"+poolingContextSource.getTimeBetweenEvictionRunsMillis());
		LOGGER.info("numTestsPerEvictionRun:"+poolingContextSource.getNumTestsPerEvictionRun());		
		LOGGER.info("minEvictableIdleTimeMillis:"+poolingContextSource.getMinEvictableIdleTimeMillis());
		

		if (ctx != null)
			ctx.close(); // Just a check so we fail on startup if its wrong.
	}

	// TODO Optimise search
	@Override
	public String search(Long msisdn, String attribute) throws CurNodeException {
		LOGGER.debug("Querying cur.");
		NamingEnumeration<SearchResult> namingEnumeration = null;
		SearchControls s = new SearchControls();
		s.setSearchScope(SearchControls.SUBTREE_SCOPE);
		s.setCountLimit(1);
		String tempValue = null;

		DirContext ctx = null;

		try {
			ctx = poolingContextSource.getContextSource().getReadOnlyContext();
			namingEnumeration = ctx.search("", "msisdn="
					+ msisdn, s);

			while (namingEnumeration.hasMore()) {
				SearchResult sr = namingEnumeration.next();
				Attributes attr = sr.getAttributes();

				Attribute tempAttribute = attr.get(attribute);
				tempValue = (tempAttribute != null) ? (String) tempAttribute
						.get() : null;

			}
			 namingEnumeration.close();

		} catch (NamingException e) {
			throw new CurNodeException(e.getClass().getSimpleName(),
					e.getMessage(), /*
									 * ldapEnv.get(ldapEnv
									 * .get("java.naming.provider.url"))
									 */"url goes here", Long.toString(msisdn));
		} finally {
			// Return Connection to pool.
			LdapUtils.closeContext(ctx);

		}
		LOGGER.debug("Querying cur finished.");
		return tempValue;
	}

}

Spring config:

	<context:property-placeholder
		location="classpath*:/META-INF/spring/cur-ldap.properties" />

	
	<bean id="curDao" class="za.co.XXX.cur.dao.CurAttributeDaoImpl"
		init-method="initialise" />



	<bean id="contextSource"
		class="org.springframework.ldap.pool.factory.PoolingContextSource">
		<property name="contextSource" ref="contextSourceTarget" />
	<!-- 	<property name="dirContextValidator" ref="dirContextValidator" /> -->
		<property name="testOnBorrow">			
			<value>${ldap.testOnBorrow}</value>
		</property>
		<property name="testWhileIdle">			
			<value>${ldap.testWhileIdle}</value>
		</property>
		<property name="maxActive">
			<value>${ldap.maxActive}</value>
		</property>
		<property name="maxIdle">
			<value>${ldap.maxIdle}</value>
		</property>
		<property name="minIdle">
			<value>${ldap.minIdle}</value>
		</property>
		<property name="maxWait">
			<value>${ldap.maxWait}</value>
		</property>
		<property name="whenExhaustedAction">
			<value>${ldap.whenExhaustedAction}</value>
		</property>
		<property name="timeBetweenEvictionRunsMillis">
			<value>${ldap.timeBetweenEvictionRunsMillis}</value>
		</property>
		<property name="numTestsPerEvictionRun">
			<value>${ldap.numTestsPerEvictionRun}</value>
		</property>
		<property name="minEvictableIdleTimeMillis">
			<value>${ldap.minEvictableIdleTimeMillis}</value>
		</property>

	</bean>

	<bean id="dirContextValidator"
		class="org.springframework.ldap.pool.validation.DefaultDirContextValidator" />

	<bean id="contextSourceTarget" class="org.springframework.ldap.core.support.LdapContextSource">
		<property name="url">
			<value>${ldap.url}</value>
		</property>
		<property name="base" >
		<value>${ldapBaseContext}</value>
		</property>
		<property name="userDn">
			<value>${ldap.aut.user}</value>
		</property>
		<property name="password">
			<value>${ldap.aut.password}</value>
		</property>
		<!-- Always set pooled to false or pool won't operate as required -->
		<property name="pooled" value="false" />
	</bean>


</beans>

spring-ldap-core : 1.3.1.RELEASE
commons-pool 1.5.6

I am using the following load test to do this, basically the first test is run, then the second just waits. I expect the beans in the context to be still alive and connections should be maintained.

public class CurAttributeDaoLoadTest {

@Autowired
CurAttributeDao dao;

@Rule
public ContiPerfRule i = new ContiPerfRule();

@Test
@PerfTest(invocations = 3500, threads = 30, rampUp = 1000, warmUp=1000)
@Required(max = 5000, average = 250)
public void testSuccess() throws NamingException, InterruptedException {
try {
String billingPlatformId = dao.search(27829822255l,
"billingPlatformId");

assertEquals("99", billingPlatformId);


} catch (CurNodeException e) {
// TODO Auto-generated catch block
e.printStackTrace();
fail(e.getMessage());
}
}

@Test
@PerfTest(invocations = 3500, threads = 30)
@Required(max = 5000, average = 250)
public void testWait() throws NamingException, InterruptedException {
// Just to keep the JVM running
Thread.sleep(5000);


}

}

Sample Output from a netstat:

Established Connections: 10
================================================== ==

Established Connections: 11
================================================== ==

Established Connections: 13

================================================== ==
<At this point, the test is waiting, the context is still alive -->
Established Connections: 0
================================================== ==

Established Connections: 0
================================================== ==

Established Connections: 0
================================================== ==

Established Connections: 0

[ericbb]

Hi,
first of all your configuration seems strange to me :

ldap.maxActive=2
ldap.maxIdle=8
ldap.minIdle=4

I think you should set maxActive to a greater value than maxIdle

Regards

[ericbb]

I also suggest that you upgrade to commons-pool 1.5.7 which fix two major bugs:
https://issues.apache.org/jira/brows...rsion/12316353