Results 1 to 4 of 4

Thread: PermGen space out of memory error

  1. #1
    Join Date
    Aug 2012
    Posts
    5

    Default PermGen space out of memory error

    Hi there,

    I am using Spring 3.1.0 and have been impressed by its capabilities.

    I tried performing a search in the forums here but was unable to get the information required.

    During development, everything was fine. Unfortunately, upon deploying to production, we came up against some issues.

    We are getting PermGen space out of memory errors.

    On our dev/test box, we have the following settings:
    -Xms2048m -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=1024m (please note that we are not confident that these are correct settings)

    On our production box, we have the following settings:
    -Xms3056m -Xmx3056m -XX:PermSize=128m -XX:MaxPermSize=512m (again, please note that we are not confident that these are correct settings)

    Admittedly, the production environment is quite different in that it has a lot more applications running on it.

    According to what I've read here, we should set the max and min to the same value and the maxpermsize to 1/4 of either one of those.

    I have read in a number of locations (including here) that it may be unavoidable to restart a server due to leaks. Is this true? This would not be possible in our environment as it is a factory. Please shed some light on this.

    Also, could you please tell me how much memory would typically be needed by the Spring framework? Upon deployment of the application, I find that the amount of memory used increases by approximately 345 MB. I understand that a number of classes need to be instantiated on startup of the application but this seems like a lot. Do I have something setup incorrectly? We are only using Spring JDBC at the moment. Is there some way to limit the framework to those classes only?

    I have seen recommendations of using

    Code:
    -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
    here and here. Are there any possible negative side effects to using these arguments?

    So, to summarise:
    1. Are the settings we have correct and, if not, what should they be?
    2. Are server restarts from time to time unavoidable with the Spring framework?
    3. How much memory should Spring be taking up? Is there a way to limit the classes used?
    4. Are there any possible side effects to using -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled ?



    Any and all help would be greatly appreciated. Thanks in advance.

    The stack trace is below (there is more but I couldn't post it all):
    Code:
    org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter]: Constructor threw exception; nested exception is java.lang.OutOfMemoryError: PermGen space
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:997)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:943)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:286)
    	at org.springframework.web.servlet.DispatcherServlet.createDefaultStrategy(DispatcherServlet.java:788)
    	at org.springframework.web.servlet.DispatcherServlet.getDefaultStrategies(DispatcherServlet.java:757)
    	at org.springframework.web.servlet.DispatcherServlet.initHandlerAdapters(DispatcherServlet.java:565)
    	at org.springframework.web.servlet.DispatcherServlet.initStrategies(DispatcherServlet.java:422)
    	at org.springframework.web.servlet.DispatcherServlet.onRefresh(DispatcherServlet.java:410)
    	at org.springframework.web.servlet.FrameworkServlet.onApplicationEvent(FrameworkServlet.java:752)
    	at org.springframework.web.servlet.FrameworkServlet$ContextRefreshListener.onApplicationEvent(FrameworkServlet.java:989)
    	at org.springframework.web.servlet.FrameworkServlet$ContextRefreshListener.onApplicationEvent(FrameworkServlet.java:1)
    	at org.springframework.context.event.GenericApplicationListenerAdapter.onApplicationEvent(GenericApplicationListenerAdapter.java:51)
    	at org.springframework.context.event.SourceFilteringListener.onApplicationEventInternal(SourceFilteringListener.java:97)
    	at org.springframework.context.event.SourceFilteringListener.onApplicationEvent(SourceFilteringListener.java:68)
    	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:97)
    	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:324)
    	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:929)
    	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:467)
    	at org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:631)
    	at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:588)
    	at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:645)
    	at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:508)
    	at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:449)
    	at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:133)
    	at javax.servlet.GenericServlet.init(GenericServlet.java:241)
    	at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:283)
    	at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    	at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
    	at weblogic.servlet.internal.StubSecurityHelper.createServlet(StubSecurityHelper.java:64)
    	at weblogic.servlet.internal.StubLifecycleHelper.createOneInstance(StubLifecycleHelper.java:58)
    	at weblogic.servlet.internal.StubLifecycleHelper.<init>(StubLifecycleHelper.java:48)
    	at weblogic.servlet.internal.ServletStubImpl.prepareServlet(ServletStubImpl.java:539)
    	at weblogic.servlet.internal.WebAppServletContext.preloadServlet(WebAppServletContext.java:1976)
    	at weblogic.servlet.internal.WebAppServletContext.loadServletsOnStartup(WebAppServletContext.java:1950)
    	at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1869)
    	at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3126)
    	at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1512)
    	at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:486)
    	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
    	at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
    	at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
    	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
    	at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
    	at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:1267)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:409)
    	at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:58)
    	at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
    	at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
    	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:569)
    	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150)
    	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116)
    	at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:143)
    	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:323)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1253)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:440)
    Last edited by the_new_mr; Aug 13th, 2012 at 01:46 PM.

  2. #2
    Join Date
    Aug 2012
    Posts
    5

    Default

    More of stack trace:
    Code:
    at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
    	at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
    	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    	at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
    
    Caused By: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter]: Constructor threw exception; nested exception is java.lang.OutOfMemoryError: PermGen space
    	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:162)
    	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:76)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:990)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:943)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:286)
    	at org.springframework.web.servlet.DispatcherServlet.createDefaultStrategy(DispatcherServlet.java:788)
    	at org.springframework.web.servlet.DispatcherServlet.getDefaultStrategies(DispatcherServlet.java:757)
    	at org.springframework.web.servlet.DispatcherServlet.initHandlerAdapters(DispatcherServlet.java:565)
    	at org.springframework.web.servlet.DispatcherServlet.initStrategies(DispatcherServlet.java:422)
    	at org.springframework.web.servlet.DispatcherServlet.onRefresh(DispatcherServlet.java:410)
    	at org.springframework.web.servlet.FrameworkServlet.onApplicationEvent(FrameworkServlet.java:752)
    	at org.springframework.web.servlet.FrameworkServlet$ContextRefreshListener.onApplicationEvent(FrameworkServlet.java:989)
    	at org.springframework.web.servlet.FrameworkServlet$ContextRefreshListener.onApplicationEvent(FrameworkServlet.java:1)
    	at org.springframework.context.event.GenericApplicationListenerAdapter.onApplicationEvent(GenericApplicationListenerAdapter.java:51)
    	at org.springframework.context.event.SourceFilteringListener.onApplicationEventInternal(SourceFilteringListener.java:97)
    	at org.springframework.context.event.SourceFilteringListener.onApplicationEvent(SourceFilteringListener.java:68)
    	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:97)
    	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:324)
    	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:929)
    	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:467)
    	at org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:631)
    	at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:588)
    	at org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:645)
    	at org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:508)
    	at org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:449)
    	at org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:133)
    	at javax.servlet.GenericServlet.init(GenericServlet.java:241)
    	at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:283)
    	at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    	at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
    	at weblogic.servlet.internal.StubSecurityHelper.createServlet(StubSecurityHelper.java:64)
    	at weblogic.servlet.internal.StubLifecycleHelper.createOneInstance(StubLifecycleHelper.java:58)
    	at weblogic.servlet.internal.StubLifecycleHelper.<init>(StubLifecycleHelper.java:48)
    	at weblogic.servlet.internal.ServletStubImpl.prepareServlet(ServletStubImpl.java:539)
    	at weblogic.servlet.internal.WebAppServletContext.preloadServlet(WebAppServletContext.java:1976)
    	at weblogic.servlet.internal.WebAppServletContext.loadServletsOnStartup(WebAppServletContext.java:1950)
    	at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1869)
    	at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3126)
    	at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1512)
    	at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:486)
    	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
    	at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
    	at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
    	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
    	at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
    	at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:1267)
    	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:41)
    	at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:409)
    	at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:58)
    	at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
    	at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
    	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:569)
    	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150)
    	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116)
    	at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:143)
    	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:323)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1253)
    	at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:440)
    	at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
    	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
    	at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
    	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    	at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
    
    Caused By: java.lang.OutOfMemoryError: PermGen space
    	at java.lang.ClassLoader.defineClass1(Native Method)
    	at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
    	at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
    	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
    	at weblogic.utils.classloaders.GenericClassLoader.defineClass(GenericClassLoader.java:328)
    	at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:285)
    	at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:253)
    	at weblogic.utils.classloaders.ChangeAwareClassLoader.findClass(ChangeAwareClassLoader.java:56)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    	at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:177)
    	at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:37)
    	at org.springframework.http.converter.StringHttpMessageConverter.<init>(StringHttpMessageConverter.java:51)
    	at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.<init>

  3. #3
    Join Date
    Aug 2012
    Posts
    2

    Default

    Hi,
    I don't think it is specific to SD project. I had PermGen problems with the applications deployed on Tomcat (MVC or Spring Integration projects, also using Quartz Sheduler), but it was usually related to some known 'Memory Leaks' offenders, or job schedulers. For example one of which was the commons http jar.

    I have learned how to view what your PermGen contains, analyze and nail down the problems by doing snapshots of the JVM during the times when the application was deployed and after is was shut down using this tool: http://www.yourkit.com/ .

    Here is a help page that will get you started : how to find memory leaks using yourkit . One day should be enough to resolve the issues.

    Sorry if I cannot be more specific. But that PermGen caused me some sleepless nights, but after some effort turned out to be fairly easy to discover and either fix, patch or avoid.

    I have done it on the Tomcat 5.5 and Tomcat 6 JVM, so not sure if this will be same under the Weblogic, but I hope it is.

    Regards,
    Dominika

  4. #4
    Join Date
    Aug 2012
    Posts
    5

    Default

    Thanks for your reply Dominika.

    I didn't use YourKit but tried JVisualVM instead (a tool that comes with the JDK). I don't think it's as detailed as YourKit in that I couldn't see what the PermGen contains but I could at least see the memory used by the heap and the perm gen memory used. I could also see the type, number and size of objects created. This showed that the main offender (with regards to heap size) seemed to be character arrays (which is fair enough considering the large amount of data handled by the application). Unfortunately, I couldn't tell what was using the perm space so much.

    We changed the production JVM settings to use 3 GB heap space and 1 GB max perm size and this has seemed to fix the problem. Here's hoping to stays that way!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •