Memory consumption in ApplicationContext 3.0.0

[ura]

Hi,

currently I'm trying to migrate my application from 2.5.6 to 3.0.0. Surprisingly I can see a significant higher memory consumption in 3.0.0. The DefaultListableBeanFactory uses 8.5 mb in spring 2.5.6 and about 90 mb in spring 3.0.0. As far as I can see, a large part of the memory consumption is due to the usage of CachingMetadataReaderFactory.

Is it possible to SimpleMetadataReaderFactory instead of CachingMetadataReaderFactory and what are the performance consequences?

Thanks in advance

Uwe

[Marten Deinum]

Hmm I would consider this a bug, could you register a JIRA with a small test case to prove the issue (shouldn't be that hard).

I'm not sure on how to switch to the SimpleMetadataReaderFactory, I suspect some dynamic discovery on which to use, not sure how to influence that.

[Costin Leau]

I've raised https://jira.springsource.org/browse/SPR-6775 We'll try to reproduce the problem but it would help to get some info or memory information about your application.

[Costin Leau]

Guys, I've committed a fix for this in the trunk, improving the internal cache of the reader with a limited, LRU.
Ura, it would be great if you could check one of the nightly builds. Out of curiosity, how big is your application and how many classes do you scan? From my tests, 90 MB is roughly equivalent to 4k classes being scanned (give or take), let's 3K which is still a lot.

[ura]

Costin, I'll try the nightly build as soon as possible...

XmlBeanDefinitionReader says:

Loaded 1257 bean definitions from location pattern [classpath*:META-INF/*-beans.xml]
Loaded 72 bean definitions from location pattern [classpath*:META-INF/spring/*.xml]
Loaded 3 bean definitions from location pattern [file:C:\Reposit.../database.xml]
Loaded 1 bean definitions from location pattern [file:C:\Reposit.../*-beans.xml]

ClassPathBeanDefinitionScanner says:

Scanning URL [jar:file:/C:/Reposit...]  2130 times

The scanning for most of the classes is done by a component-scan element like this:

<context:component-scan base-package="de.varial.accounting.persistency,ce.varial.accounting.persistency,com.varial.accounting.persistency" resource-pattern="**/*Dao.class" />

Most of the classes scanned have a structure like the example below:

@Repository(value="xyzDao")
public final class XyzDao implements RowMapper<XyzContainer>, IXyzDao {

	@Autowired
	private final PrimaryKeyIncrementer incrementer = null;
	
	@Autowired
	@Qualifier("dataSource")
	private final DataSource dataSource = null; 

	@Autowired
	private final SQLExceptionTranslatorFactory translator = null;

	@Autowired
	private final StatementOptimizer optimizer = null;
		
	public XyzDao() {
		..
	}

	@Override
    public  XyzContainer getInitializedInstance() {
		..
    }

	@Override
	public  void getRows(A, B, C) {
		..
	}

	@Override
	public  void getRows(A, B, C, D) {
		...
	}

	private boolean putData(A, B) {
		...
	}

	@Override
	public  XyzContainer insert(A, B) {
		...
	}

	@Override
	public  XyzContainer update(A, B) {
		...
	}

	@Deprecated
	@Override
	public  XyzContainer updateIgnoreVersion(A, B) {
		...
	}

	@Override
	public  void updateAttributes(A, B) {
		...
	}

	@Override
	public  int updateAttributes(A, B, C) {
		...
	}

	@Override
	public void delete(A, B) {
		...
	}

	@Override
	public void delete(A, B, C) {
		...
	}

	@Override
	public int delete(A, B) {
		...
	}

	private  PreparedStatement fillUpdateParameter(A, B) {
		...
	}

	private  PreparedStatement fillInsertParameter(A, B) {
		...
	}
    
	private  List<String> getContainerAsStrings(A, B) {
		...
	}

	@Override
	public  XyzContainer getRowByObjectID(A) {
		...
	}

	@Override
	public  XyzContainer[] getRows(A, B) {
		...
	}

	private  XyzContainer makeContainer(A) {
		...
	}

	@Override
	public  long getRowCount(A) {
		...
	}

	@Override
	public  PersistencyDataSet getDataSet(A, B) {
		...
	}

	@Override
	public XyzContainer mapRow(A, B) {
		...
	}

	private SQLExceptionTranslator getExceptionTranslator() {
		...
	}
}

[ura]

Just an additional information...

I've attached 2 ZIP files containing GIFs. The file Memory-2.5.6.gif shows the memory consumption using spring 2.5.6, the file Memory-3.0.0.gif shows the same information using spring 3.0.0. I'll do the same for the nightly build, mentioned in the posts before, alter on.

[Costin Leau]

You have a pretty big application - around 1500 definitions (let's assume one class per definition) and the scanner introspects over 2K classes so all in all, we're well over 3K classes analyzed which adds up to the figures I've seen from my tests.

P.S. the latest snapshot (562) has been already published ~12h ago so it should be ready for download

[ura]

Costin, I did the test with 3.0.1.CI-562. The results are not good but much better compared to 3.0.0.RELEASE. In 3.0.0.RELEASE the memory consumption of CachingMetadataReaderFactory was about 81 mb, in 3.0.1.CI-562 the consumption is reduced to 18 mb (see the attached jpg).

[Costin Leau]

Hmm - It should be less then that. I'm using YourKit and the integration test (which goes through 10k classes) doesn't even show on the profiler anymore and the memory usage never goes beyond 8MB. I'll try Eclipse MAT and maybe a more complicated class - maybe your classes are bigger then the one I use which might explain the difference in memory size. We currently use 256 entries so we could lower this number to 128 which should reduce the size to half of what it's now.

[Costin Leau]

Out of curiosity, could you check the average size of your classes in actual bytes on the file system - I think this might be the most important aspect.