Sending a message to multiple endpoints with transforms.

[Mark Fisher]

David,

We do plan to include an OXM-based MessageTransformer implementation in M5. Feel free to watch, vote, or add any comments to this JIRA issue: http://jira.springframework.org/browse/INT-109

If you'd like to open another issue for XSLT-based transformation, please do.

Regards,
Mark

[iwein]

Before I go off topic again, is the solution summarized by Marius workable for you Dave?

There are cases when the transformation occurs on the message itself (either payload or header). The typical example for that is Content Enricher. On another note, the identity of the processed message might be necessary to be preserved for a Claim Check based handling ...

It is perfectly possible to store a key in the message, use that to retrieve the data during the claim and create a new message with the data in it. There is no reason as far as I can see to require message identity preservation.

Relying on identity would be a bad idea imho. The whole point is loose coupling and if you don't know the route of the message you can't be sure that identity is preserved. Some component might persist messages in queues or send them over the network for example.

I'm not saying that mutable messages should be illegal or something, my only point is that I can't see a good reason you would want to use them. This brings us back to the transformer interface.

The MessageTransformer interface has been designed for this mutable Message case, and that's why transform() does not return anything.

It would be no problem to allow both mutable messages an immutable messages if we would return the message from the transform() method. I think that the interface needs to be changed. It doesn't have any advantages to return void.

Anyway, maybe I'm just addicted to immutable objects...

[mbogoevici]

]
There is no reason as far as I can see to require message identity preservation.

The reason for being able to preserve message identity is:

http://www.eaipatterns.com/MessageStore.html

implemented as

org.springframework.integration.message.MessageSto re.

]
It would be no problem to allow both mutable messages an immutable messages if we would return the message from the transform() method. I think that the interface needs to be changed. It doesn't have any advantages to return void.

OK, let's make sure we're on the same page here: first, by discussing the fundamental difference between a Transformer and a Handler: a Transformer modifies the existing message, whereas the Handler grabs the message and potentially returns a new one (it may not return anything, for that matter).

If we would return something from the transform() method, there'd be no difference with a MessageHandler (please compare the two interfaces). Now, that'd be just duplicating existing functionality, which would be nonsensical, right ? So, the difference must be there for a reason, which is allowing the users to mutate the messages. There's no talk here on what's preferable in any given situation, it is what the framework supports or not.

If you want to use a strict immutable message-based processing model, which, in particular, I very much agree with, just use MessageHandlers instead of MessageTransformers for your handling, that's the whole point. Please understand their different purpose and different functionality.


Regards,
Marius

[iwein]

OK, let's make sure we're on the same page here: first, by discussing the fundamental difference between a Transformer and a Handler: a Transformer modifies the existing message, whereas the Handler grabs the message and potentially returns a new one (it may not return anything, for that matter).

If we would return something from the transform() method, there'd be no difference with a MessageHandler (please compare the two interfaces). Now, that'd be just duplicating existing functionality, which would be nonsensical, right ? So, the difference must be there for a reason, which is allowing the users to mutate the messages. There's no talk here on what's preferable in any given situation, it is what the framework supports or not.

If you want to use a strict immutable message-based processing model, which, in particular, I very much agree with, just use MessageHandlers instead of MessageTransformers for your handling, that's the whole point. Please understand their different purpose and different functionality.

I do understand, but I still have some arguments with it. In any case we can take it offline. There is a clear understanding of Dave's problem, a solution and a workaround. My further arguments are irrelevant to this thread.

[mbogoevici]

In any case we can take it offline. There is a clear understanding of Dave's problem, a solution and a workaround.

Well, we walked a bit out of the main topic, as I said before. But I hope that it helped in explaining the fundamental differences between MessageTransformer and MessageHandler, as implemented by M4. And of course, let's hope it helps us build a better M5 .

[david_geary]

Iwein,

When you say marius' solution do you mean?

But I'm afraid we moved away from Dave's original issue: this can be solved relatively easy now, by introducing supplemental Endpoints before the e-mail and webservice Targets, which would take charge of the transformation in a MessageHandler - rather than using a MessageTransformer - so the original shared copy is not altered, but rather a new message is created every time. Of course, this should just give some relief until the cloning option will be available.

Currently I'm running the message through a clone handler in a chain with the transformer handler adapter to get round the problem. If I did the above I wouldn't be using the transformer interface anymore which I want to try to stick with as I have my own oxm,xslt and xpath implementations of it and this seems more consistent with where the next milestone release is going.

For me doing it either way - either cloning in the pub / sub channel and multi channel routers or having transformers behave more like handlers would work.

Actually originally I was using a handler to do the transform - but wanted to move to the new standard transformer interface and that's when I hit the problem

Although immutable messages is the simpler fix, I can see it's more flexible to have the framework distinguish between transformers and handlers.

Another option would be to put a clone option on the TransformerMessageHandlerAdapter?

Dave

[mbogoevici]

Hi Dave,

Indeed, the idea is to do transformation on a separate copy. Interface-related, the end goal is to be able to use pojos for transformation/handling, case in which switching from one to the other is a matter of modifying an annotation.

The Transformer model will definitely undergo changes in the M5, but this snippet from the unit tests should be illustrative:

private static class HandlerWithTransformers {

		@Transformer
		@Order(-1)
		public String transformBefore(String input) {
			return "pre." + input;
		}

		@Handler
		@Order(0)
		public String handle(String input) {
			return input.toUpperCase();
		}

		@Transformer
		@Order(1)
		public String transformAfter(String input) {
			return input + ".post";
		}
	}

Unfortunately, there's no namespace equivalent at this time, yet.

We plan on addressing the issue of cloning messages in a pub-sub scenario before M5.


Regards,
Marius

[iwein]

Iwein,
When you say marius' solution do you mean (quote omitted)?

Yes.

If I did the above I wouldn't be using the transformer interface anymore which I want to try to stick with as I have my own oxm,xslt and xpath implementations of it and this seems more consistent with where the next milestone release is going.

For me doing it either way - either cloning in the pub / sub channel and multi channel routers or having transformers behave more like handlers would work.

Actually originally I was using a handler to do the transform - but wanted to move to the new standard transformer interface and that's when I hit the problem

Although immutable messages is the simpler fix, I can see it's more flexible to have the framework distinguish between transformers and handlers.

One of my points was that we could distinguish and offer the option of doing a copy on transform (by adding a return type to the method). The downside that Marius pointed out is that we couldn't be sure of the identity preservation of the message anymore in that scenario.

One other important point of Marius that I missed is that it doesn't work that way now hence, if you want it to change, you should create an issue for it.

At the moment we're waiting for Mark to shed his light on this discussion (which will take a until after next week). If you want to move forward on the immutable path you could easily implement MessageHandler first and refactor later (to the better method name). You could also glue with something like...

abstract class ImmutableMessageTransformer implements MessageHandler{..}

...which is pretty horrible but very easy to refactor once MessageTransformer is changed (if it ever will be). It's still a bit dodgy, but that means it's good we have this discussion before RC1.

[david_geary]

Thanks for the feedback,

Indeed, the idea is to do transformation on a separate copy. Interface-related, the end goal is to be able to use pojos for transformation/handling, case in which switching from one to the other is a matter of modifying an annotation.

Off topic slightly, but one point to note is that what we are trying to do with spring integration is provide a customisable event publishing mechanism for our product, where we publish event messages with a known (published schema) xml payload on fixed message channels. The integration people will then write custom spring configuration files that get added into the spring context to transform and route the messages to do what ever they need to do.

Hence we are trying to stay away from custom POJOs (although they may occasionally be necessary) and in particular annotations as we are trying to rely on supplied transformers such as xslt and xpath (eg to pull out a bit of content for emails)

Hence the namespace support would be important to us (to make it easier for the integration people to configure) whereas the annotation stuff isn't really at present.

I'm interested to see what the namespace support for transformers / transformers chains will look like. In particular the code you show seems to show the application of pre and post transformers to a message handler. If a pre transformer could be added to message targets also then this could provide a very concise way to make web service calls ie something like

<channel id="eventpublishingchannel" publish-subscribe="true" clone="true"/>

<ws-target id="bpelTarget" 
		uri="http://localhost:8081/bpel/services/TestService" 
		channel="eventpublishingchannel"
		transformer="bpelRequestTransformer"/>

<xslt-transformer id="bpelRequestTransformer" xslFile="C:\bpelRequest.xsl"/>

In this example the eventpublishingchannel would be the publish subscribe channel with the clone support as already discussed enabling multiple endpoints using the events on this channel.

The transforms could also be done in the target-endpoint tag.

Dave

[mbogoevici]

David,

Thanks for your feedback as well - it's very helpful for us. Namespace support is critical from our point of view as well, it's just that it needs to be ready for M5, as M4 had to be released before we had that part ready.

For target endpoints, it would look something like:

<channel id="eventpublishingchannel" publish-subscribe="true" clone="true"/>

<target-endpoint input-channel="exampleChannel" target="bpelTarget" 
    transformer="bpelRequestTransformer">
    <schedule period="#somePeriod"/>
</target-endpoint>

<ws-target id="bpelTarget" 
		uri="http://localhost:8081/bpel/services/TestService" 
		channel="eventpublishingchannel"/>

<xslt-transformer id="bpelRequestTransformer" xslFile="C:\bpelRequest.xsl"/>

It's pretty much what you wrote, except that transformers will be placed on the endpoints themselves. Also, if there's more than one transformer per endpoint (i.e. a chain), they could be placed as sub-elements of <*-endpoint/>, the order being relevant in this case.

Cannot say 100% that this will be the final syntax, but this is by and large how it will look like.

The intent is for the transformers will be hosted in an endpoint (just like handlers) but, in principle, to operate on the same message (again, besides payload handling it can be a matter of header/envelope manipulation). In the case of a target endpoint, it is just pre-processing, in the case of a source-endpoint, it is just post-processing, and for a handler-endpoint they would provide either pre-processing or postprocessing, depending on their placement relative to the handlers.

Regards,
Marius