Message received via POP3 and placed into RabbitMQ has empty Payload data

[ehrdoctors]

I am new to SI and after reading some examples, I am trying to write a program of my own that seems simple enough. The program attempts to read an email message via POP3 and put the message body on a RabbitMQ. It seemed to be working (there is a message on the queue) until I configured RabbitMQ Management so that I could view the Messages and much to my disappointment the payload is EMPTY!

The server reported 1 messages remaining.
Exchange si.test.exchange
Routing Key si.test.binding
Redelivered ●
Properties
priority: 0
delivery_mode: 2
headers:
content_type: application/octet-stream
Payload 0 bytes Encoding: string

Please help as I have tried various variations (chains, different channels, etc) of the code with the same results.

Here is my spring config:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
		http://www.springframework.org/schema/integration/mail http://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd
		http://www.springframework.org/schema/integration/file http://www.springframework.org/schema/integration/file/spring-integration-file-2.1.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
		http://www.springframework.org/schema/integration/amqp http://www.springframework.org/schema/integration/amqp/spring-integration-amqp.xsd
		http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd" 
	xmlns:int-amqp="http://www.springframework.org/schema/integration/amqp"
	xmlns:int="http://www.springframework.org/schema/integration"
	xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
	xmlns:util="http://www.springframework.org/schema/util"
	xmlns:rabbit="http://www.springframework.org/schema/rabbit"
	xmlns:si="http://www.springframework.org/schema/integration"
	xmlns:file="http://www.springframework.org/schema/integration/file"
	>

	<int:channel id="incomingMail" >
    	</int:channel>

	<int:channel id="outgoingRabbit" >
	 <int:interceptors>
            <int:wire-tap channel="loggingChannel"/>
        </int:interceptors>
    	</int:channel>

	<int:logging-channel-adapter id="loggingChannel" log-full-message="true" level="DEBUG"/>

	<!-- replace 'userid and 'password' wit the real values -->
	<int-mail:inbound-channel-adapter id="pop3ShouldDeleteTrue" 
	                    	     store-uri="pop3://#########:#########@my.mailhost.com/INBOX" 
			            channel="incomingMail" 
			            should-delete-messages="true" 
			            auto-startup="true" 
			            java-mail-properties="javaMailProperties">
		
		<!-- Will poll every 20 seconds -->            
		<int:poller fixed-rate="20000"/>
			
	</int-mail:inbound-channel-adapter>


	<si:chain input-channel="incomingMail" output-channel="outgoingRabbit">
		<si:transformer expression="payload.content"/>
	</si:chain>


	<int-amqp:outbound-channel-adapter channel="outgoingRabbit"
		amqp-template="amqpTemplate" exchange-name="si.test.exchange"
		routing-key="si.test.binding"/>


	<util:properties id="javaMailProperties">
		<prop key="mail.pop3.socketFactory.fallback">false</prop>
		<prop key="mail.debug">true</prop>
		<prop key="mail.pop3.port">995</prop>
		<prop key="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
		<prop key="mail.pop3.socketFactory.port">995</prop>
	</util:properties>


	<!-- Infrastructure -->
    	<rabbit:connection-factory id="connectionFactory" /> 
    	<rabbit:template id="amqpTemplate" connection-factory="connectionFactory" />
    	<rabbit:admin connection-factory="connectionFactory" />
    	<rabbit:queue name="si.test.queue" />
    	<rabbit:direct-exchange name="si.test.exchange">
        	<rabbit:bindings>
            		<rabbit:binding queue="si.test.queue" key="si.test.binding"/>
        	</rabbit:bindings>
    	</rabbit:direct-exchange>

</beans>

[Gary Russell]

The default converter (SimpleMessageConverter) will create an empty payload if the data is not byte[], String, or Serializable.

I suspect your payload.content expression is not resulting in a String as you expected. Depending on how the original mail message is constructed, you may need to use getBodyPart(...) on the content and then do further transformation as necessary.

If you run with debug logging, you should see the payload type coming out of the transformer (chain).

[Gary Russell]

See the JavaDoc for MimeMessage.getContent() for more information....

...
     * Return the content as a Java object. The type of this
     * object is dependent on the content itself. For 
     * example, the native format of a "text/plain" content
     * is usually a String object. The native format for a "multipart"
     * message is always a Multipart subclass. For content types that are
     * unknown to the DataHandler system, an input stream is returned
     * as the content. <p>
     *
     * This implementation obtains the content from the DataHandler,
     * that is, it invokes <code>getDataHandler().getContent()</code>.
     * If the content is a Multipart or Message object and was created by
     * parsing a stream, the object is cached and returned in subsequent
     * calls so that modifications to the content will not be lost.
...

[ehrdoctors]

See the JavaDoc for MimeMessage.getContent() for more information....

Right you are Gary, I ended up changing my <chain> to a <transformer> like this:

  <int:transformer input-channel="incomingMail" output-channel="outgoingRabbit" ref="MailTransformer" method="transformPayload"/>
        <bean id="MailTransformer" class="org.mdh.pop3amqp.MailTransformer"/>

I also added a MailTransformer class with a transformPayload method like this. It parses a multi part MIME or a plain/text message, and puts the payload on a String.

package org.mdh.pop3amqp;
import javax.mail.internet.MimeMessage;
import javax.mail.*;
import javax.mail.internet.*;
import org.springframework.integration.transformer.AbstractPayloadTransformer;

public class MailTransformer extends AbstractPayloadTransformer<MimeMessage, String> {
                @Override
                protected String transformPayload(MimeMessage message) throws Exception {
                StringBuilder ret = new StringBuilder(0);
                try {

                        if(message.getContent() instanceof Multipart){
                                Multipart multipart = (Multipart) message.getContent();

                                for (int x = 0; x < multipart.getCount(); x++) {
                                        BodyPart bodyPart = multipart.getBodyPart(x);
                                        String disposition = bodyPart.getDisposition();
                                        ret.append(bodyPart.getContent());
                                }
                        }
                        else{ //Assume its a String
                                ret.append(message.getContent());
                        }
                }
                catch(Exception e){
                        return "Error Parsing MIME Message";
                }

                return ret.toString();
        }
}

It works, the queue now shows:

The server reported 0 messages remaining.
Exchange si.test.exchange
Routing Key si.test.binding
Redelivered ○
Properties
priority: 0
delivery_mode: 2
headers:
content_encoding: UTF-8
content_type: text/plain
Payload 14 bytes Encoding: string

test payload

[ehrdoctors]

Now that I have this working, I need a strategy to to:

1. Add the MIME Message Headers to the Message destined for the Rabbit Queue. I am not sure how to do this. I saw a AbstractHeaderTransformer() class http://static.springsource.org/sprin...ansformer.html but I am not sure how to use it with what I already have...

2. Deal with attachments. The code above can be changed to identify attachments in the bodyPart like this:

 
String disposition = bodyPart.getDisposition();
                                        if (disposition != null && (disposition.equals(BodyPart.ATTACHMENT))) {
                                                DataHandler handler = bodyPart.getDataHandler();
                                        }
                                        else{
                                                ret.append(bodyPart.getContent());
                                        }

The question is how should the attachment be sent to the RabbitMQ. What formats have you seen in an integration parttern that goes between Mail and RabbitMQ? I know XML and JSON are possible options. I am not sure how this would be implemented in the transformPayload method I created to parse out the message body. I guess I could format the XML string manually and base64encode the attachment. Any advise would be appreciated.

[Gary Russell]

You can map any headers you want, using the mapped-request-headers attribute on the amqp adapter, e.g...

mapped-request-headers="foo*"

There's no standard way to map a Java MimeMessage to an AMQP message; you're pretty much on your own. If you come up with a reasonable general solution, feel free to submit it for consideration as an extension... https://github.com/SpringSource/spri...ion-extensions