Results 1 to 10 of 10

Thread: Shutting down Flex Message Broker (cleanly)

  1. #1

    Default Shutting down Flex Message Broker (cleanly)

    Hi,

    I'm trying to hunt down why my Spring/Flex app doesn't unload all it's classes properly when the web-app context is reloaded (or even stopped).

    I'm seeing the following error in the Tomcat logs:

    Code:
    SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1d9d7ac]) and a value of type [flex.messaging.MessageBroker] (value [flex.messaging.MessageBroker@1717854]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

    Basically, all I have done is start the web-app, then stop it (using the Tomcat manager)

    I'm using a basic Flex message broker (using the namespace):

    Code:
    <flex:message-broker>
    		<flex:exception-translator ref="rtExceptionHandler" />
    		<flex:message-service
    			default-channels="longpolling-amf" />
    		<flex:secured />
    	</flex:message-broker>
    Is this a known issue that the message broker doesn't clean up properly on shutdown? Is there a work around?

    Thanks in advance,

    Kevin.

    p.s. using Tomcat 6.0.26 & the 'find leaks' feature.

  2. #2
    Join Date
    Apr 2005
    Location
    San Francisco, CA
    Posts
    1,224

    Default

    As far as I'm aware, we're doing everything necessary to try and ensure that all of the BlazeDS ThreadLocals are cleaned up properly when the Spring ApplicationContext is destroyed. I supposed it could be possible that the code we are calling in BlazeDS to do this is not doing the right thing. I have not tried running it with the "find leaks" support myself, but will give it a try to see if there is some way we can improve this from our end.
    Jeremy Grelle

    Staff Engineer, Web Products Team
    SpringSource

  3. #3
    Join Date
    Oct 2007
    Posts
    17

    Default Same problem...

    +1

    Shudown my app, then issued tomcat http://localhost:8080/manager/findleaks for tomcat 6.0.26, and got:

    Code:
        May 27, 2010 11:38:21 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@bfba23]) and a value of type [org.springframework.flex.core.MessageBrokerFactoryBean.DelegatingServletConfig] (value [org.springframework.flex.core.MessageBrokerFactoryBean$DelegatingServletConfig@1557525]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        May 27, 2010 11:38:21 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@17de242]) and a value of type [flex.messaging.config.SystemSettings] (value [flex.messaging.config.SystemSettings@1917b9e]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.
        May 27, 2010 11:38:21 AM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap
        SEVERE: A web application created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@4af41d]) and a value of type [flex.messaging.MessageBroker] (value [flex.messaging.MessageBroker@e0a75a]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed.

    I did attach a debugger, and verified that on application shutdown the spring MessageBrokerFactoryBean destroy() method is being invoked. That in turn calls FlexContext.clearThreadLocalObjects(), but I don't have source for that. It definitely looks like FlexContext.clearThreadLocalObjects isn't doing the right thing.

    I'm using blazeds version: 3.2.0.3978
    and spring-flex 1.0.0.RELEASE

  4. #4
    Join Date
    Oct 2007
    Posts
    17

    Default Tried one more thing, to no avail...

    On application shutdown, I manually called FlexContext.releaseThreadLocalObjects(), but still got the same find leaks error messages.

  5. #5
    Join Date
    Oct 2007
    Posts
    17

    Default And tried 1.0.3.RELEASE, to no avail...

    Tried the newest spring-flex release (1.0.3.RELEASE), but it exhibits the same findleaks behavior described above.

    Thx,
    Brad

  6. #6
    Join Date
    Oct 2010
    Posts
    1

    Default

    Are there any news or progress on the shutdown memory leak?

  7. #7
    Join Date
    Apr 2005
    Location
    San Francisco, CA
    Posts
    1,224

    Default

    Well, it is an issue with BlazeDS itself. I haven't checked to see if any of the later (unofficial) 3.0.x builds fix the problem. FWIW, I do know they've cleaned up the code responsible for this a good bit in BlazeDS 4.0.x. I haven't had a chance to check it thoroughly against Tomcat's leak detection though.
    Jeremy Grelle

    Staff Engineer, Web Products Team
    SpringSource

  8. #8
    Join Date
    Oct 2007
    Posts
    17

    Default Aha - Found the source of the memory leak

    Below analysis is for spring-flex 1.0.3.RELEASE and BlazeDS 4.0.0.14931

    1. On the main 'startup' thread during application startup, the <flex:message-broker id="_messageBroker"/> tag creates and initializes the BlazeDS MessageBroker. As a side-effect of creating the MessageBroker, several ThreadLocals for the startup thread are created:[ DelegatingServletConfig, SystemSettings, MessageBroker, SerializationContext ]

    2. The problem is that these thread locals on the startup are never cleared. Thus when Tomcat shuts the application down, it complains about the thread leak (see original error msg in this post).

    3. The fix is fairly simple: in my 'blazeds-config.xml' file used for the blazeds servlet configuration, I added:

    Code:
      <!-- CleanupBlazeDSThreadLocalsOnStartup uses @PostConstruct in this servlet context -->
      <context:annotation-config/>
    
    and 
    
     <!--
           Spring flex does NOT cleanup BlazeDS thread locals created from main initialization thread,
           which results in Tomcat reporting a memory leak at shutdown.
      -->
      <bean class="com.abinitio.dashboard.dispatch.flex.utils.CleanupBlazeDSThreadLocalsOnStartup"
            depends-on="_messageBroker"/>
    The CleanupBlazeDSThreadLocalsOnStartup bean is also constructed on the 'startup' thread, and thus can clear the corresponding thread locals that are left over from the message broker creation :
    Code:
      @PostConstruct
      public void afterPropertiesSet()
      {
        // Clear the thread locals used by the 'main' startup thread.
        // Otherwise get thread leak messages from Tomcat on application shutdown.
        FlexContext.clearThreadLocalObjects();
    
        // Clear other thread local objects on the 'main' initialization thread
        SerializationContext.clearThreadLocalObjects();
        TypeMarshallingContext.clearThreadLocalObjects();
      }

    This worked - after doing the above, Tomcat no longer complains about BlazeDS-related memory leaks.

  9. #9
    Join Date
    Oct 2007
    Posts
    17

  10. #10
    Join Date
    Apr 2005
    Location
    San Francisco, CA
    Posts
    1,224

    Default

    Thanks for digging into this and for the detailed analysis...I believe everything should be getting cleaned up properly now in 1.5.
    Jeremy Grelle

    Staff Engineer, Web Products Team
    SpringSource

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •