That's a tricky one - are you saying that sometimes replyQueue contains replies, and sometimes just unsolicited messages?

Given that the gateway uses a selector looking for the JMSMessageID in the JMSCorrelationID, you could, for unsolicited messages, have the other side set JMSCorrelationID to, say "foo" and have a <int-jms:message-driven-channel-adapter ... selector="JMSCorrelationID='foo'" .../>.