Page 1 of 2 12 LastLast
Results 1 to 10 of 19

Thread: Memory consumption in ApplicationContext 3.0.0

  1. #1
    Join Date
    Jan 2006
    Location
    Siegen, Germany
    Posts
    12

    Default Memory consumption in ApplicationContext 3.0.0

    Hi,

    currently I'm trying to migrate my application from 2.5.6 to 3.0.0. Surprisingly I can see a significant higher memory consumption in 3.0.0. The DefaultListableBeanFactory uses 8.5 mb in spring 2.5.6 and about 90 mb in spring 3.0.0. As far as I can see, a large part of the memory consumption is due to the usage of CachingMetadataReaderFactory.

    Is it possible to SimpleMetadataReaderFactory instead of CachingMetadataReaderFactory and what are the performance consequences?

    Thanks in advance

    Uwe

  2. #2
    Join Date
    Jun 2006
    Location
    The Netherlands
    Posts
    13,625

    Default

    Hmm I would consider this a bug, could you register a JIRA with a small test case to prove the issue (shouldn't be that hard).

    I'm not sure on how to switch to the SimpleMetadataReaderFactory, I suspect some dynamic discovery on which to use, not sure how to influence that.
    Marten Deinum
    Java Consultant / Pragmatist / Open Source Enthousiast / Author


    Pro Spring MVC: With Web Flow
    Conspect

    Have you read the reference guide.
    Use the [ code ] tags, young padawan

  3. #3
    Join Date
    Jan 2005
    Location
    Bucharest, Romania
    Posts
    5,403

    Default

    I've raised https://jira.springsource.org/browse/SPR-6775 We'll try to reproduce the problem but it would help to get some info or memory information about your application.
    Costin Leau
    SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
    http://twitter.com/costinl
    Please use [ c o d e ] [ / c o d e ] tags

  4. #4
    Join Date
    Jan 2005
    Location
    Bucharest, Romania
    Posts
    5,403

    Default

    Guys, I've committed a fix for this in the trunk, improving the internal cache of the reader with a limited, LRU.
    Ura, it would be great if you could check one of the nightly builds. Out of curiosity, how big is your application and how many classes do you scan? From my tests, 90 MB is roughly equivalent to 4k classes being scanned (give or take), let's 3K which is still a lot.
    Costin Leau
    SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
    http://twitter.com/costinl
    Please use [ c o d e ] [ / c o d e ] tags

  5. #5
    Join Date
    Jan 2006
    Location
    Siegen, Germany
    Posts
    12

    Default

    Costin, I'll try the nightly build as soon as possible...

    XmlBeanDefinitionReader says:

    Code:
    Loaded 1257 bean definitions from location pattern [classpath*:META-INF/*-beans.xml]
    Loaded 72 bean definitions from location pattern [classpath*:META-INF/spring/*.xml]
    Loaded 3 bean definitions from location pattern [file:C:\Reposit.../database.xml]
    Loaded 1 bean definitions from location pattern [file:C:\Reposit.../*-beans.xml]
    ClassPathBeanDefinitionScanner says:

    Code:
    Scanning URL [jar:file:/C:/Reposit...]  2130 times
    The scanning for most of the classes is done by a component-scan element like this:

    Code:
    <context:component-scan base-package="de.varial.accounting.persistency,ce.varial.accounting.persistency,com.varial.accounting.persistency" resource-pattern="**/*Dao.class" />
    Most of the classes scanned have a structure like the example below:

    Code:
    @Repository(value="xyzDao")
    public final class XyzDao implements RowMapper<XyzContainer>, IXyzDao {
    
    	@Autowired
    	private final PrimaryKeyIncrementer incrementer = null;
    	
    	@Autowired
    	@Qualifier("dataSource")
    	private final DataSource dataSource = null; 
    
    	@Autowired
    	private final SQLExceptionTranslatorFactory translator = null;
    
    	@Autowired
    	private final StatementOptimizer optimizer = null;
    		
    	public XyzDao() {
    		..
    	}
    
    	@Override
        public  XyzContainer getInitializedInstance() {
    		..
        }
    
    	@Override
    	public  void getRows(A, B, C) {
    		..
    	}
    
    	@Override
    	public  void getRows(A, B, C, D) {
    		...
    	}
    
    	private boolean putData(A, B) {
    		...
    	}
    
    	@Override
    	public  XyzContainer insert(A, B) {
    		...
    	}
    
    	@Override
    	public  XyzContainer update(A, B) {
    		...
    	}
    
    	@Deprecated
    	@Override
    	public  XyzContainer updateIgnoreVersion(A, B) {
    		...
    	}
    
    	@Override
    	public  void updateAttributes(A, B) {
    		...
    	}
    
    	@Override
    	public  int updateAttributes(A, B, C) {
    		...
    	}
    
    	@Override
    	public void delete(A, B) {
    		...
    	}
    
    	@Override
    	public void delete(A, B, C) {
    		...
    	}
    
    	@Override
    	public int delete(A, B) {
    		...
    	}
    
    	private  PreparedStatement fillUpdateParameter(A, B) {
    		...
    	}
    
    	private  PreparedStatement fillInsertParameter(A, B) {
    		...
    	}
        
    	private  List<String> getContainerAsStrings(A, B) {
    		...
    	}
    
    	@Override
    	public  XyzContainer getRowByObjectID(A) {
    		...
    	}
    
    	@Override
    	public  XyzContainer[] getRows(A, B) {
    		...
    	}
    
    	private  XyzContainer makeContainer(A) {
    		...
    	}
    
    	@Override
    	public  long getRowCount(A) {
    		...
    	}
    
    	@Override
    	public  PersistencyDataSet getDataSet(A, B) {
    		...
    	}
    
    	@Override
    	public XyzContainer mapRow(A, B) {
    		...
    	}
    
    	private SQLExceptionTranslator getExceptionTranslator() {
    		...
    	}
    }

  6. #6
    Join Date
    Jan 2006
    Location
    Siegen, Germany
    Posts
    12

    Default

    Just an additional information...

    I've attached 2 ZIP files containing GIFs. The file Memory-2.5.6.gif shows the memory consumption using spring 2.5.6, the file Memory-3.0.0.gif shows the same information using spring 3.0.0. I'll do the same for the nightly build, mentioned in the posts before, alter on.
    Attached Files Attached Files

  7. #7
    Join Date
    Jan 2005
    Location
    Bucharest, Romania
    Posts
    5,403

    Default

    You have a pretty big application - around 1500 definitions (let's assume one class per definition) and the scanner introspects over 2K classes so all in all, we're well over 3K classes analyzed which adds up to the figures I've seen from my tests.

    P.S. the latest snapshot (562) has been already published ~12h ago so it should be ready for download
    Costin Leau
    SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
    http://twitter.com/costinl
    Please use [ c o d e ] [ / c o d e ] tags

  8. #8
    Join Date
    Jan 2006
    Location
    Siegen, Germany
    Posts
    12

    Default

    Costin, I did the test with 3.0.1.CI-562. The results are not good but much better compared to 3.0.0.RELEASE. In 3.0.0.RELEASE the memory consumption of CachingMetadataReaderFactory was about 81 mb, in 3.0.1.CI-562 the consumption is reduced to 18 mb (see the attached jpg).
    Attached Files Attached Files

  9. #9
    Join Date
    Jan 2005
    Location
    Bucharest, Romania
    Posts
    5,403

    Default

    Hmm - It should be less then that. I'm using YourKit and the integration test (which goes through 10k classes) doesn't even show on the profiler anymore and the memory usage never goes beyond 8MB. I'll try Eclipse MAT and maybe a more complicated class - maybe your classes are bigger then the one I use which might explain the difference in memory size. We currently use 256 entries so we could lower this number to 128 which should reduce the size to half of what it's now.
    Costin Leau
    SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
    http://twitter.com/costinl
    Please use [ c o d e ] [ / c o d e ] tags

  10. #10
    Join Date
    Jan 2005
    Location
    Bucharest, Romania
    Posts
    5,403

    Default

    Out of curiosity, could you check the average size of your classes in actual bytes on the file system - I think this might be the most important aspect.
    Costin Leau
    SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
    http://twitter.com/costinl
    Please use [ c o d e ] [ / c o d e ] tags

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •