JDBCMessageStore and MimeMailMessage

[alexbarnes]

I have created a MailGateway for my application which uses a JDBCMessageStore to back to queue:

@Gateway
public void sendMail(OutgoingMailMessage recipient);

and configured it as follows:

<channel id="outboundMailChannel" >
  <queue message-store="messageStore" />
</channel>
<jdbc:message-store data-source="dataSource" id="messageStore" />

This used to send SimpleMailMessage instances perfectly well. I have now found that to send HTML e-mail I need to use MimeMailMessage. However, I now get the following exception when I try to send an e-mail:

Failed to serialize object using DefaultSerializer; nested exception is java.io.NotSerializableException: org.springframework.mail.javamail.MimeMailMessage

This makes sense because SimpleMailMessage is Serializable and MimeMailMessage is not. I'd like to know if anyone has any ideas to allow this to work? I can fix it for now by removing the JDBCMessageStore but am looking for something better!

[Mark Fisher]

The first thing that comes to mind is that you can reference any implementation of the Spring Serializer interface via the 'serializer' attribute on the JDBC Message Store configuration element. That way, you can choose an alternative to the standard Java serialization (which is obviously the default). Of course, you'd need to implement the symmetric Deserailzer behavior.

This is one area that is really calling out for extensions (implementations of those strategies). We hope to have several out-of-the-box options at some point, similar to the way Spring OXM supports so many different XML marshalling technologies. Of course, community contributions are more than welcome!

Thanks,
Mark

[Amol Nayak]

You can provide a serializer and deserializer attributes to the message-store something like

<jdbc:message-store data-source="dataSource" id="messageStore" serializer="serializer"  deserializer="deserializer"/>

serializer and deserializer are beans of class implementing the org.springframework.core.serializer.Serializer org.springframework.core.serializer.Deserializer interfaces respectively. You need to provide your custom implementation there

[alexbarnes]

Thanks both for your prompt replies. I did try replacing the default Serializer with my own implementation. However, the issue here is that the MimeMailMessage doesn't implement Serializable.

This was my attempt. Note that it just circumvents the check for the Serializable interface!

public void serialize(Object object, OutputStream outputStream)
			throws IOException {
  ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
  objectOutputStream.writeObject(object);
  objectOutputStream.flush();
}

However, this doesn't work because the MimeMailMessage isn't Serializable. Do you have any suggestions for the actual implementation of my Serializer? I'm very happy to contribute the code back to the community if I write one and it's useful! Perhaps you could point me in the right direction...

[Amol Nayak]

That is exactly why we use serializer, you *donot* serialize the MimeMailMessage . This is painful but you need to extract the fields you are interested in like the from, to, bcc, subject, content etc and store it in a object that is serializable. On deserialization you recreate a MimeMailMessage based on these values stored.

But as mark pointed out, this must indeed be a common scenario and we can have some serializer and deserializer supported out of the box. Feel free to raise a JIRA for this enhancement here

[alexbarnes]

Sounds simple enough. I'll get started. Thanks.

[Mark Fisher]

Amol's right. What you showed above is basically the same as what you'd see in Spring's DefaultSerializer:

/**
 * Writes the source object to an output stream using Java Serialization.
 * The source object must implement {@link Serializable}.
 */
public void serialize(Object object, OutputStream outputStream) throws IOException {
	if (!(object instanceof Serializable)) {
		throw new IllegalArgumentException(getClass().getSimpleName() + " requires a Serializable payload " +
				"but received an object of type [" + object.getClass().getName() + "]");
	}
	ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
	objectOutputStream.writeObject(object);
	objectOutputStream.flush();
}

...and that's the source of the problem, since the object you pass is not Serializable.

There is one other option. Rather than a Serializer, you could add something like object-to-json transformer upstream. I don't know off the top of my head if that would work with MMM out of the box, but it can be customized (its ObjectMapper) and that might end up being less work overall. Not sure if you're interested in that approach, but I figured I'd throw it out there.

Other serialization options we would like to have in the future are protocol buffers, thrift, avro, etc.

[alexbarnes]

The DefaultSerializer is exactly where I got the code from for mine! I just removed the 'object instanceof Serializable' part to see what would happen. The result? Not much. Not suprising.

Thanks for all your suggestions. I'll have a play with it later.

P.S. Was very impressed with how easy it was to get the whole thing working with SimpleMailMessage. JDBC backed queue with polling etc.

[alexbarnes]

Both,

I've had a go at implementing a Serializer and Deserializer for a MimeMailMessage. I am getting an e-mail delivered which is good news however, I am seeing odd behaviour in the logs. When I was using a simple mail message with the DefaultSerializer after my e-mail had been sent the TRACE logs went back to showing a poll finding no messages. I would guess that the messages were removed from the database once sent?

Now I am seeing a continuous stream of logs:

TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
DEBUG: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
TRACE: org.springframework.integration.monitor.SimpleMessageHandlerMetrics - messageHandler(org.springframework.integration.mail.MailSendingMessageHandler#0) message([Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]) :
DEBUG: org.springframework.integration.mail.MailSendingMessageHandler - org.springframework.integration.mail.MailSendingMessageHandler#0 received message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Removing message from group with group key=9ea43c5e-5587-3754-ada7-99f7801195f2
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
DEBUG: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
TRACE: org.springframework.integration.monitor.SimpleMessageHandlerMetrics - messageHandler(org.springframework.integration.mail.MailSendingMessageHandler#0) message([Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]) :
DEBUG: org.springframework.integration.mail.MailSendingMessageHandler - org.springframework.integration.mail.MailSendingMessageHandler#0 received message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Removing message from group with group key=9ea43c5e-5587-3754-ada7-99f7801195f2
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2

My GenericMessage headers are the same in my Serializer and Deserializer suggesting that they are correctly serialized and deserialized. My headers before and after are as follows:

{timestamp=1330642786346, id=62452ca5-d827-4ef3-8d72-3f0e761541ea, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}
{timestamp=1330642786346, id=62452ca5-d827-4ef3-8d72-3f0e761541ea, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}

Does anything obvious spring to mind?

My Serializer is as follows:

public void serialize(GenericMessage<MimeMailMessage> object,
			OutputStream outputStream) throws IOException {MimeMailMessageStore store = new     MimeMailMessageStore("test@test.com", "Test", "Test",object.getHeaders());

// -- Serialize the storage object
ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
objectOutputStream.writeObject(store);
objectOutputStream.flush();

Where the MimeMailMessageStore is an immutable class which implements Serializable and stores the recipient address, subject, content and message headers.

My deserializer is as follows:

@Inject
	private MimeMailMessage mailMessage;

	public GenericMessage<MimeMailMessage> deserialize(InputStream inputStream)
			throws IOException {
		ObjectInputStream objectInputStream = new ObjectInputStream(inputStream);
		
		/*// -- Attempt to recreate our message
		try {
			mailMessage.getMimeMessageHelper().setTo("test@test.com");
			mailMessage.getMimeMessageHelper().setSubject("Test");
			mailMessage.getMimeMessageHelper().setText("Text", true);
			
			MimeMailMessageStore store = (MimeMailMessageStore) objectInputStream.readObject();
			
			Map<String, Object> headers = new HashMap<String, Object>(store.getHeaders());
			headers.put(MessageHeaders.ID, store.getHeaders().get(MessageHeaders.ID));
			
			return new GenericMessage<MimeMailMessage>(
					mailMessage, headers);
		} catch (Exception e) {
			throw new RuntimeException(e);
		}
		
	}

where the MimeMailMessage is an injected prototype scoped bean.

Your thoughts and tips would be appreciated!

[alexbarnes]

I have dug a bit more and now I a little confused:

I am creating a GenericMessage<MimeMailMessage> when I deserialize. I believe that I want to put the id in the header back to what it was before? Correct?

However, if I construct a GenericMessage passing in a Map<String, Object> containing the headers the GenericMessage constructor will use this Map to construct a new MessageHeaders. The constructor on MessageHeaders will set the ID value regardless. Therefore we can never set it despite this comment:

/**
	 * The key for the Message ID. This is an automatically generated UUID and
	 * should never be explicitly set in the header map <b>except</b> in the
	 * case of Message deserialization where the serialized Message's generated
	 * UUID is being restored.
	 */
	public static final String ID = "id";

Have I missed something here?

I am using version 2.1.0.RELEASE.