component-scan is really slow

[adambrod]

Hi-

I am experience a big delay starting my container because of component-scanning. We have a pretty large application with thousands of classes and dozens of jar files. The component-scanning part of initialization takes about 1-2 mins on my windows box. (I have a good box with fast disks.)

I have configured my application context to explicitly scan a handful of packages just within my company's source code. However, I noticed that the scanner searches the entire classpath, including all jars. Is it possible to narrow the scope so it only looks in the source directory?

I know that Seam used to require you to put a seam.properties file in the classpath directory or jar as a marker telling seam to scan that classpath. Is there anything like that I can do?

Any advice is much appreciated.

-Adam

[Marten Deinum]

spring scans the whole classpath (if you would like this changed register a JIRA issue). Apart from being more selective on what to scan there isnt much you can do about it. If you post your configuration we might find something to tweak.

Also I wouldn't worry to much about startup times of an application (at least when it is a server application). Although it can be a pain when testing.

[adambrod]

I'm mostly concerned about the startup time for our integration tests. We have a lot of them and waiting 2 mins for it is start is pretty frustrating.

Below are some snippets from my applicationContext.xml. Originally I was going to just scan for com.mycompany but I ran our of heap space, so I switched to many component-scan elements.

    <!-- We have to force cglib proxies or else cobertura screws up the proxying -->
    <aop:aspectj-autoproxy proxy-target-class="true"/>
    <!-- This must be picked up first -->        
    <context:component-scan base-package="com.mycompany.container"/>    
    
    <!-- Put base packages below to be scanned (don't add subpackages as they are picked up already) -->
    <context:component-scan base-package="api.mycompany.catalog"/>
    <context:component-scan base-package="com.mycompany.ads"/>
    <context:component-scan base-package="com.mycompany.api"/>
    <context:component-scan base-package="com.mycompany.affiliate"/>
    <context:component-scan base-package="com.mycompany.authentication"/>
    <context:component-scan base-package="com.mycompany.beta"/>
    <context:component-scan base-package="com.mycompany.catalog"/>
    <context:component-scan base-package="com.mycompany.cache"/>
   // this goes on - we have about 15 packages being scanned

Thanks for your help!

[Marten Deinum]

Well 2 minutes for your integration tests is still faster then 1 or 2 testers integration testing your application . But I get your concern.

1 thing you could do is reduce the number of component-scan elements saves you instantiation time of each element. the base-package attribute can take a comma separated list of packages. That probably gives you a bit of a performance gain.

<context:component-scan base-package="com.mycompany.container,api.mycompany.catalog,com.mycompany.ads....../>

The scanning of the classpath is hardcoded into the ClassPathScanningCandidateComponentProvider so no way of influencing that. You might consider registering a JIRA to make this configurable (scan all (classpath* or limited (classpath: ).

Thats the best I can come up with for now.

[adambrod]

FYI, specifying multiple base-classpath's does not work. You can specify multiple component-scan elements though.

[adambrod]

I did some profiling and I don't think the component scanning is as big of a performance penalty as I thought. I wrote some code to generate an applicationContext.xml using the annotations and my application didn't start any faster. Now I think it might be the AOP stuff..

[mystic]

In my experience, and I don't know what you're architecture is, but hibernate tends to take the most time out of anything. If you're not using hibernate, it other orms can take some time to do their thing too. If your app takes 10-20 seconds to start up, that seems acceptable.

Also, the comma-separated list does in fact work. I'm using that now.

[adambrod]

Thanks for replying, mystic. I am surprised that you can get the comma separated list working. I currently have about 10 directories, so maybe my problem was that I put each directory on its own line and that caused a problem...

Hibernate does take about 20 to start in our system, but I'm seeing more like 70 seconds to initialize the container context... I wish I knew where it was all going. I did some profiling, but nothing too obvious jumped out - there were lots of lots of Spring methods that were called but didn't take too long.

[mystic]

What other things are you running? You gotta remember too, anything static is going to take some time as well I think, so it might not be anything you are instantiating through the container. How much logging have you turned on? Crank it