Framework-dependent Import-package when developing a bundle

[dlkpochtaws]

We are developing an application using Spring DM.
We are creating bundles by maven's maven-bundle-plugin. It is declared to calculate Import-package clause for resulting bundle.

But many frameworks (especially spring and DM) have very complicated logic which cannot be understood by bnd when it calculates Import-packages.
So, our import-package in pom.xml looks like this:


<Import-Package>
org.springframework.transaction.interceptor;resolu tion:=optional,
org.springframework.jdbc.support;resolution:=optio nal,
org.springframework.jdbc.datasource;resolution:=op tional,
org.springframework.beans.factory.config;resolutio n:=optional,
org.springframework.beans.factory.xml;resolution:= optional,
org.springframework.aop;resolution:=optional,
org.springframework.aop.framework;resolution:=opti onal,
org.springframework.aop.framework.autoproxy;resolu tion:=optional,
org.springframework.context.annotation;resolution: =optional,
org.springframework.dao.support;resolution:=option al,
org.springframework.core.io.support;resolution:=op tional,
org.springframework.osgi.service.importer.support; resolution:=optional,
org.springframework.osgi.service.exporter;resoluti on:=optional,
org.springframework.osgi.service.exporter.support; resolution:=optional,
org.springframework.orm.jpa;resolution:=optional,
org.springframework.orm.jpa.support;resolution:=op tional,
org.springframework.jndi;resolution:=optional,
org.springframework.util;resolution:=optional,
org.aopalliance.aop;resolution:=optional,
org.h2;resolution:=optional,
javax.jdo.listener;resolution:=optional,
javax.persistence;resolution:=optional,
org.apache.xerces.impl.dv.dtd;resolution:=optional ,
org.apache.xerces.impl.dv.xs;resolution:=optional,
*;resolution:=optional
</Import-Package>


All those packages are added one by one, when we debug an application, and see logging like this:


*** Class 'org.springframework.util.Assert' was not found because bundle 15 does not import 'org.springframework.util' even though bundle 2 does export it. To resolve this issue, add an import for 'org.springframework.util' to bundle 15. ***

As far as I understand from debugger, this concrete message is caused by spring DM's extender. When our bundle is started, extender (extender's Activator) is trying to verify some a condition by calling smth like:


Assert.isTrue()

but instead of check extender has got an ClassNotFoundException. I do not know, can extender handle this situation, so I add


org.springframework.util;resolution:=optional,

And I repeat this procedure until there are no ClassNotFound exceptions in log.

But the given import is not for my code, like most other imports shown above - it is for framework code, and I can not be sure I've added all required imports - there are other execution paths in framework.
I think it is not a maintainable approach to add every implicitly-required package to Import-package clause.
1) I can not document why the concrete line is added.
2) Those lines can not be refactored by IDE.
3) There are too many of them.

So, my question is - Is there a more smart approach for adding "implicitly required" packages?

[dlkpochtaws]

BTW, we are using Spring DM v 1.1.1 and Spring 2.5.5

[Costin Leau]

Hi,



As far as I understand from debugger, this concrete message is caused by spring DM's extender


To be correct, this message can be caused by any type of classloading and, by the looks of your log, it's Felix that actually does the logging (other frameworks might just throw the exception).
Spring and Spring-DM themselves are OSGi bundles out of the box so I'm not sure why you'd like to add imports for them since these are already present. Can you clarify?

As for the usage of the bnd plugin, I recommend you read their page: http://felix.apache.org/site/maven-bundle-plugin-bnd.html.
For cases as you described, using a wildcard should pick up the dependency and add the import - afterall, the reason of using the plugin is to automatically pick up the imports rather then request them.

[dlkpochtaws]

Hi Costin, after your explanations I've downloaded extender's sources and watched the stacktrace.
The stacktrace is:


java.lang.ClassNotFoundException: *** Class 'org.springframework.util.Assert' was not found because bundle 15 does not import 'org.springframework.util' even though bundle 2 does export it. To resolve this issue, add an import for 'org.springframework.util' to bundle 15. ***
at org.apache.felix.framework.searchpolicy.R4SearchPo licyCore.findClass(R4SearchPolicyCore.java:195)
at org.apache.felix.framework.searchpolicy.R4SearchPo licy.findClass(R4SearchPolicy.java:45)
at org.apache.felix.moduleloader.ModuleImpl.getClass( ModuleImpl.java:152)
at org.apache.felix.framework.Felix.loadBundleClass(F elix.java:1476)
at org.apache.felix.framework.BundleImpl.loadClass(Bu ndleImpl.java:341)
at org.springframework.osgi.extender.internal.activat or.SpringTypeCompatibilityChecker.isTypeAvailable( SpringTypeCompatibilityChecker.java:107)
at org.springframework.osgi.extender.internal.activat or.SpringTypeCompatibilityChecker.checkCompatibili ty(SpringTypeCompatibilityChecker.java:64)
at org.springframework.osgi.extender.internal.activat or.ContextLoaderListener.maybeCreateApplicationCon textFor(ContextLoaderListener.java:721)
at org.springframework.osgi.extender.internal.activat or.ContextLoaderListener$ContextBundleListener.han dleEvent(ContextLoaderListener.java:230)
at org.springframework.osgi.extender.internal.activat or.ContextLoaderListener$BaseListener.bundleChange d(ContextLoaderListener.java:173)
at org.apache.felix.framework.util.EventDispatcher.in vokeBundleListenerCallback(EventDispatcher.java:69 0)


So, as far as I understand, this my example above is not a problem, but feature - extender tries to verify if spring classes are available. In this case I can do not react to this warning. There are a lot of such warnings, BTW. I can not guess which is just a feature and which is a problem. I must download sources and trace them. But it is a general Felix problem, I believe :-)

But I can give other example. If I remove import:


org.springframework.orm.jpa.support;resolution:=op tional,

I get the exception:


org.springframework.beans.factory.UnsatisfiedDepen dencyException: Error creating bean with name 'filterDao' defined in URL [bundle://17.0:1/org/test/dao/jpox/FilterDaoImpl.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.jdo.PersistenceManagerFactory]: : Cannot find class [org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor] for bean with name 'org.springframework.context.annotation.internalPe rsistenceAnnotationProcessor' defined in null; nested exception is java.lang.ClassNotFoundException: org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor not found from bundle [org.zekarkass.zekarkass-runtime-ui]; nested exception is org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor] for bean with name 'org.springframework.context.annotation.internalPe rsistenceAnnotationProcessor' defined in null; nested exception is java.lang.ClassNotFoundException: org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor not found from bundle [org.zekarkass.zekarkass-runtime-ui]
at org.springframework.beans.factory.support.Construc torResolver.createArgumentArray(ConstructorResolve r.java:591)
at org.springframework.beans.factory.support.Construc torResolver.autowireConstructor(ConstructorResolve r.java:193)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.autowireConstructor(Abs tractAutowireCapableBeanFactory.java:925)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBeanInstance(Abst ractAutowireCapableBeanFactory.java:835)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:440)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:409)
at java.security.AccessController.doPrivileged(Native Method)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:380)
at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 64)
at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:221)
at org.springframework.beans.factory.support.Abstract BeanFactory.doGetBean(AbstractBeanFactory.java:261 )
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:185)
at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:164)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.preInstantiateSingletons(Defaul tListableBeanFactory.java:429)
at org.springframework.context.support.AbstractApplic ationContext.finishBeanFactoryInitialization(Abstr actApplicationContext.java:729)
at org.springframework.osgi.context.support.AbstractD elegatedExecutionApplicationContext.completeRefres h(AbstractDelegatedExecutionApplicationContext.jav a:276)
at org.springframework.osgi.extender.internal.depende ncies.startup.DependencyWaiterApplicationContextEx ecutor$CompleteRefreshTask.run(DependencyWaiterApp licationContextExecutor.java:145)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor] for bean with name 'org.springframework.context.annotation.internalPe rsistenceAnnotationProcessor' defined in null; nested exception is java.lang.ClassNotFoundException: org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor not found from bundle [org.zekarkass.zekarkass-runtime-ui]
at org.springframework.beans.factory.support.Abstract BeanFactory.resolveBeanClass(AbstractBeanFactory.j ava:1138)
at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.predictBeanType(Abstrac tAutowireCapableBeanFactory.java:524)
at org.springframework.beans.factory.support.Abstract BeanFactory.isFactoryBean(AbstractBeanFactory.java :1174)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.getBeanNamesForType(DefaultList ableBeanFactory.java:222)
at org.springframework.beans.factory.BeanFactoryUtils .beanNamesForTypeIncludingAncestors(BeanFactoryUti ls.java:187)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.findAutowireCandidates(DefaultL istableBeanFactory.java:652)
at org.springframework.beans.factory.support.DefaultL istableBeanFactory.resolveDependency(DefaultListab leBeanFactory.java:610)
at org.springframework.beans.factory.support.Construc torResolver.resolveAutowiredArgument(ConstructorRe solver.java:622)
at org.springframework.beans.factory.support.Construc torResolver.createArgumentArray(ConstructorResolve r.java:584)
... 17 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor not found from bundle [org.zekarkass.zekarkass-runtime-ui]
at org.springframework.osgi.util.BundleDelegatingClas sLoader.findClass(BundleDelegatingClassLoader.java :103)
at org.springframework.osgi.util.BundleDelegatingClas sLoader.loadClass(BundleDelegatingClassLoader.java :156)
at java.lang.ClassLoader.loadClass(ClassLoader.java:2 51)
at org.springframework.util.ClassUtils.forName(ClassU tils.java:242)
at org.springframework.beans.factory.support.Abstract BeanDefinition.resolveBeanClass(AbstractBeanDefini tion.java:383)
at org.springframework.beans.factory.support.Abstract BeanFactory.resolveBeanClass(AbstractBeanFactory.j ava:1135)
... 25 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor
at org.apache.felix.framework.Felix.loadBundleClass(F elix.java:1479)
at org.apache.felix.framework.BundleImpl.loadClass(Bu ndleImpl.java:341)
at org.springframework.osgi.util.BundleDelegatingClas sLoader.findClass(BundleDelegatingClassLoader.java :99)
... 30 common frames omitted


The problem here is: Spring DM creates an app context, then Spring tries to create a post processor for an app context, but the class is not visible to my bundle. BND plugin can not help here, because is parses only .java classes but do not parse spring's .xml files and do not analyze spring's postprocessors.
As a result, I must import framework-specific classes by hands.

[Costin Leau]

Dmitry, I'm still not sure what you are trying to do... The extender doesn't try to test Spring classes - it really uses them. The Assert for example is used internally throughout the code. Felix throws the warning since probably the extender bundle doesn't start properly - however it should since the distribution as well as the sources have a proper manifest.
This leads me to believe there is something wrong with your environment.
As for Spring and the post processor - what post processor are you talking about? In most cases these are already loaded so there is nothing for you to worry.
However you are right that BND doesn't parse the XML files (I believe there is a plugin for this) so any class defined in there needs to visible (imported) as well in the bundle.

[dlkpochtaws]

Dmitry, I'm still not sure what you are trying to do... The extender doesn't try to test Spring classes - it really uses them. The Assert for example is used internally throughout the code. Felix throws the warning since probably the extender bundle doesn't start properly - however it should since the distribution as well as the sources have a proper manifest.

Talking about extender testing Spring classes I mean class SpringTypeCompatibilityChecker. Lines 61-74


/** Spring class used for loading */
private static Class SPRING_TYPE = Assert.class;

boolean checkCompatibility(Bundle bundle) {
Assert.notNull(bundle);

Boolean typeComparison = isTypeAvailable(bundle, SPRING_TYPE);

if (typeComparison != null) {
return typeComparison.booleanValue();
}
// check the imported bundles
else {
ImportedBundle[] importedBundles = dependencyResolver.getImportedBundles(bundle);
return checkImportedBundles(importedBundles);
}
}


Here extender ensures Spring classes are visible to our bundle. If our bundle does not import Spring package explicitly, they are not visible. By default we do not Import-package Spring's packages (after all Spring is not invasive, our bundles do not import any spring class), so Felix writes a warning message to the console when lines of code above are executed.
When I see Felix's warning "ClassNotFound" I prefer to import the package in question, since I do not know, whether the caller is prepared to ClassNotFoundException. But now I see we must revise this conception - not all Felix's warning are really warnings, Spring DM is ready for ClassNotFoundException :-)
So I agree, this case is not a problem.




But we can look at another example. We have a bundle with Spring app context. The context contains a DAO. We must add a line to Import-package


<Import-Package>
org.springframework.jdbc.support;resolution:=optio nal,
</Import-Package>

If this line is absent, there is a message in the log:


ResourceNotFoundException org/springframework/jdbc/support/sql-error-codes.xml
WARN o.s.j.s.SQLErrorCodesFactory - Default sql-error-codes.xml not found (should be included in spring.jar)

This is because Spring during the creation of database-access classes initializes a SQLErrorCodesFactory class.
There is a code:


/**
* The name of default SQL error code files, loading from the class path.
*/
public static final String SQL_ERROR_CODE_DEFAULT_PATH = "org/springframework/jdbc/support/sql-error-codes.xml";

/**
* Keep track of a single instance so we can return it to classes that request it.
*/
private static final SQLErrorCodesFactory instance = new SQLErrorCodesFactory();

protected SQLErrorCodesFactory() {
Map errorCodes = null;

try {
DefaultListableBeanFactory lbf = new DefaultListableBeanFactory();
XmlBeanDefinitionReader bdr = new XmlBeanDefinitionReader(lbf);

// Load default SQL error codes.
Resource resource = loadResource(SQL_ERROR_CODE_DEFAULT_PATH);
if (resource != null && resource.exists()) {
bdr.loadBeanDefinitions(resource);
}
else {
logger.warn("Default sql-error-codes.xml not found (should be included in spring.jar)");
}
.............skip..........
}
catch (BeansException ex) {
logger.warn("Error loading SQL error codes from config file", ex);
errorCodes = Collections.EMPTY_MAP;
}

this.errorCodesMap = errorCodes;
}

The warning is caused by line 110:


Resource resource = loadResource(SQL_ERROR_CODE_DEFAULT_PATH);


As we can see, Spring can not load some defaults. I think this is a real warning and must be corrected. So I correct it by adding framework-specific import to my bundle.
And this a problem, I suppose.

[dlkpochtaws]

As for Spring and the post processor - what post processor are you talking about? In most cases these are already loaded so there is nothing for you to worry.


I mean Spring's class AnnotationConfigUtils:



/**
* The bean name of the internally managed JPA annotation processor.
*/
public static final String PERSISTENCE_ANNOTATION_PROCESSOR_BEAN_NAME =
"org.springframework.context.annotation.internalPer sistenceAnnotationProcessor";
private static final String PERSISTENCE_ANNOTATION_PROCESSOR_CLASS_NAME =
"org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor";
private static final boolean jpaPresent =
ClassUtils.isPresent("javax.persistence.EntityManagerFactory", AnnotationConfigUtils.class.getClassLoader()) &&
ClassUtils.isPresent(PERSISTENCE_ANNOTATION_PROCES SOR_CLASS_NAME, AnnotationConfigUtils.class.getClassLoader());


public static Set<BeanDefinitionHolder> registerAnnotationConfigProcessors(
BeanDefinitionRegistry registry, Object source) {
...skip....
// Check for JPA support, and if present add the PersistenceAnnotationBeanPostProcessor.
if (jpaPresent && !registry.containsBeanDefinition(PERSISTENCE_ANNOT ATION_PROCESSOR_BEAN_NAME)) {
RootBeanDefinition def = new RootBeanDefinition();
def.setBeanClassName(PERSISTENCE_ANNOTATION_PROCES SOR_CLASS_NAME);
def.setSource(source);
def.setRole(BeanDefinition.ROLE_INFRASTRUCTURE);
beanDefinitions.add(registerBeanPostProcessor(regi stry, def, PERSISTENCE_ANNOTATION_PROCESSOR_BEAN_NAME));
}
...skip.....


In our case, we use Spring's "<context:annotation-config/>" in bundle's application context, so the AnnotationConfigBeanDefinitionParser is created. And this parser calls (line 46)


Set<BeanDefinitionHolder> processorDefinitions =
AnnotationConfigUtils.registerAnnotationConfigProc essors(parserContext.getRegistry(), source);


AnnotationConfigUtils register a post processor for bean factory (see the code above), of class named "org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor". So this class must instantiated during app context init, and if our bundle does not import org.springframework.orm.jpa.support, the bundle can not be loaded.
Again, our code does not refer to org.springframework.orm.jpa.support package in any way, so I must import it into the pom.xml by hands.

[Costin Leau]

Dmitry, you are right about the compatibility check - this is done so if a different version of Spring is used by your bundle then by the extender (rare case but can happen), a proper logging message is given.
Note that there are cases where the bundle doesn't use Spring (valid case) which is just fine - in this case Felix through a warning but again, this is harmless.
Regarding loading of resources, that requires an import (non-optional I would say) - however for your particular case this can be improved (see http://jira.springframework.org/browse/SPR-5118).

[dlkpochtaws]

Thank you Costin.
By the way, I think the problem with org.springframework.orm.jpa.support.PersistenceAnn otationBeanPostProcessor is more serious than SQLErrorCodesFactory warning.

[Costin Leau]

The processor issue has been raised internally but before addressing it, we have to see whether the changes don't create other issues as well. Currently this forces a, somewhat, implicit import which is acceptable in my opinion.