OK, here are some numbers in bytes..
Size of bytecode: arithmetic mean 25842, min 11920, max 61478, standard deviation 4935
Printable View
OK, here are some numbers in bytes..
Size of bytecode: arithmetic mean 25842, min 11920, max 61478, standard deviation 4935
Now I had a deeper look into the code...
As far as I can see the obfuscated class ...asm.ClassReader, which is indirectly referenced from the cache of CachingMetadataReaderFactgory, holds the bytecode of a bean class and an array of strings. This array holds, among other stuff, stringified method signatures defined in or called from the bean class.
For my application, I found this string array mostly larger then the bytecode array (see attachment) and short snippets below.
Code:([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;Lcom/varial/base/persistency/callback/IConsumer<Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;>;I)V
([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;Lcom/varial/base/persistency/callback/IConsumer<Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;>;)V
(Ljava/util/List<Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;>;Lcom/varial/base/persistency/callback/IConsumer<Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;>;)Z
Ljava/lang/Object;Lorg/springframework/jdbc/core/RowMapper<Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;>;Lcom/varial/migrationprocess/persistency/migpwwag/IMigpwwagDao;
(Ljava/sql/Connection;Ljava/lang/String;[Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;I)Ljava/sql/PreparedStatement;
(Ljava/sql/Connection;Ljava/lang/String;[Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;)Ljava/sql/PreparedStatement;
([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;)[Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;
(Ljava/lang/String;Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;)Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;
([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;Lcom/varial/base/persistency/callback/IConsumer;I)V
([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;)Lcom/varial/base/persistency/PersistencyDataSet;
([Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;Lcom/varial/base/persistency/callback/IConsumer;)V
(Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;[Ljava/lang/String;Lcom/varial/base/persistency/criterion/Criterion;)I
(Ljava/lang/String;[Lcom/varial/base/persistency/criterion/Order;Lcom/varial/base/persistency/criterion/Criterion;)Ljava/lang/String;
(Ljava/sql/PreparedStatement;Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;)Ljava/sql/PreparedStatement;
(Ljava/sql/Connection;Ljava/sql/ResultSet;Ljava/sql/PreparedStatement;Lorg/springframework/jdbc/core/RowMapper;)V
(Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;Z)Ljava/util/List<Ljava/lang/String;>;
(Ljava/lang/String;Ljava/lang/String;Ljava/sql/SQLException;)Lorg/springframework/dao/DataAccessException;
(Ljava/sql/ResultSet;I)Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;
(Ljava/lang/String;Lcom/varial/base/persistency/criterion/Criterion;)Ljava/lang/String;
(Ljava/sql/ResultSet;)Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;
(Ljava/lang/String;)Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;
(Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;Z)Ljava/util/List;
(Ljava/lang/String;Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;)V
(Lcom/varial/migrationprocess/persistency/migpwwag/MigpwwagContainer;[Ljava/lang/String;)V
(Ljavax/sql/DataSource;)Lorg/springframework/jdbc/support/SQLExceptionTranslator;
Hi ura,
Thanks for looking into this. I've fixed this problem already however I'd like to minimize the impact of the cache even further (not sure if that's possible but I'm looking into it).
For example, if the classes have a big number of methods (40) the amount of metadata can add up so I'm trying to compact any wasted space.
If compiling the trunk is an option, I can commit what I have right now assuming you can then update it on your side. Otherwise, I'll spend some more time and then fire up a nightly build to publish a new snapshot for you to try.
How does it sound?
Hi Costin,
firing up a nightly build after finishing your changes will be perfectly working for me. Pls give me a short message after doing the nightly build. I'll try the shapshot afterwards.
Thanks
Uwe
Uwe, I've improved the way information is stored inside the cache which should be overall, a lot more efficient now. When dealing with a LOT (hundreds) of huge classes (with plenty of annotated methods (more then 20-30)), the cache becomes big but in your case, it should drop to 20% of what it is right now, maybe more.
Let me know how it works for you - we might consider introducing a time based policy so that the cache expires in time to improve memory for long running applications.
If that doesn't work, then probably using a dedicated MetadataReader or the simple one is better though if the latter is used there would be _significant_ IO activity.
Anyway, build 563 has been started (http://build.springframework.org/bro...-TRUNKSNAPSHOT) - it should be ready in about 90 minutes from now.
Looking forward to your feedback!
Uwe the snapshot has been published - please let us know how it works for you.
Costin, I've made a first test... You really did a great job, now the memory consumption seems to be more or less comparable to sprint 2.5.6. I'll do some more tests on monday afternoon and give you the results afterwards.
Have a nice weekend
Uwe
Glad to hear this Uwe. I'd appreciate if you could do some profiling on your app and see how it compares overall to 2.5.6. We're trying to identify the hotspots and getting feedback from the community in various scenarios helps a lot.
Hi Costin, below you'll find the detailed numbers for my application (approx. in mb):
Once again, great job...Code:Spring 2.5.6
DefaultListableBeanFactory 8.7
beanDefinitionMap 4.8
mergedBeanDefinitions 2.1
singletonObjects 0.008
Spring 3.0.0.RELEASE
DefaultListableBeanFactory 97.5
beanDefinitionMap 12.2
mergedBeanDefinitions 2.1
singletonObjects 81.3
Spring 3.0.1.CI-562
DefaultListableBeanFactory 34.9
beanDefinitionMap 12.4
mergedBeanDefinitions 2.1
singletonObjects 18.4
Spring 3.0.1.CI-563
DefaultListableBeanFactory 10.1
beanDefinitionMap 5.3
mergedBeanDefinitions 2.1
singletonObjects 1.0
Spring 3.0.1.CI-565
DefaultListableBeanFactory 10.1
beanDefinitionMap 5.3
mergedBeanDefinitions 2.1
singletonObjects 1.0
Uwe