Is this a good candidate for SI?

[dheinecke]

Hi Spring community,

I have written a reverse-proxy of a sort using Spring MVC. It works, but the code that generates the downstream requests seems clunky. An engineer here suggested that we look at SI as a solution to clean it up and make it more extensible. Here is how it works:

An http request comes in to our server. The payload of the request is xml, but it is opaque to the proxy. The proxy is only concerned with the request path and the request parameters. Using the path and parameters, it calculates which downstream server to route the request to. It then sends the request (different path, identical parameters and payload) to the DSS. In some cases, it will route the request to multiple DSSs. Once the http responses from the DSS(s) are gathered, it returns an http xml response to the original request.

As I mentioned earlier, we have this working already using Spring MVC (although we are awaiting 3.2 so we can use async MVC), so I'd like to know if this would be something fairly easy to do in SI, or whether we should stick to plain old MVC.

Note: I'm pretty new to the SI model; I've done a couple of simple "Hello world"-style SI apps, but despite extensive googling, I haven't found an example out there that is sufficiently similar to what we're trying to do. Any pointers or opinions would be fantastic!

Thanks!

[Gary Russell]

There was a recent discussion about this very subject http://forum.springsource.org/showth...-proxy-pattern that talks about various options.

You should be able to do what you need as-is, but we do have an open JIRA to handle stuff like automatically propagating soap action headers etc... https://jira.springsource.org/browse/INT-2466 rather than requiring explicit configuration.

[oleg.zhurakousky]

Also, it would be good to have a code somewhere in Github so we can look at it. We are also planning to open up a community extension project where things like this could live.

[dheinecke]

Thanks for the advice and the links.

I spent some time on this today and this is what I came up with. It mostly works, but I'm running into a few problems.

First, the channel and gateway definitions:

    <int:channel id="inboundChannel"></int:channel>
    <int:channel id="outboundChannel"></int:channel>
    <bean id="podActivator" class="com.mcafee.orion.globaldirector.spring.PodServiceActivator">
        <property name="channel" ref="outboundChannel"/>
    </bean>
    <http:inbound-gateway request-channel="inboundChannel" name="/{subject}" path="/{subject}" supported-methods="GET,POST">
        <http:header name="subject" expression="#pathVariables.subject"/>
    </http:inbound-gateway>
    <http:outbound-gateway request-channel="outboundChannel" url="{outBoundGatewayUrl}" http-method="POST"
                           expected-response-type="java.lang.String"
                           reply-timeout="5000" charset="UTF-8">
        <http:uri-variable name="outBoundGatewayUrl" expression="headers['X_REQUEST_URL']"/>
    </http:outbound-gateway>
    <int:service-activator id="notification.activator" input-channel="inboundChannel" ref="podActivator"/>

I figured a ServiceActivator was the best way to get my custom logic to determine where to dispatch the request to. For my simple test, I am just printing the message and then dispatching to a well-known URL.

@Component
public class PodServiceActivator {

    private MessageChannel channel;

    @ServiceActivator
    public void receive( Message<?> message )
    {
        // send URL was http://localhost:8080/notification/c...&action=create
        System.out.println("Http message: " + message );
        System.out.println(message.getHeaders().get("subject"); // customer -- yay!
        LinkedMultiValueMap m = (LinkedMultiValueMap) message.getPayload();
        Message msg = MessageBuilder.withPayload( "<notification xmlns=" + m.get("<notification xmlns").get(0) )
                .setHeader( "X_REQUEST_URL", "http://localhost:8080/bps/customer?accountNumber=9999&action=create")
                .setHeader( "Content-Type", "application/text" ).build();
        channel.send( msg );
    }

    public void setChannel( MessageChannel channel )
    {
        this.channel = channel;
    }
}

This kinda works but there are a few problems (some misconfigurations and misunderstandings on my part, surely )

The first is that the message is a LinkedMultiValueMap with three entries: 1->accountNumber, 2->action, and 3->my xml payload... unfortunately, when I put the message back into the channel, it comes out on the DSS all glommed together. To counter this, I have to generate a new message and parse out the xml element, but it is keyed as a string ("<notification xmlns="). Perhaps I need to specify a different mime-type?

The second problem is that when it does dispatch to the DSS, the URI '?' is escaped so the values can't be parsed correctly there...

/bps/customer%3FaccountNumber=9999&action=create)

This last issue seems to be due to a problem with UriTemplate as discussed here:
http://forum.springsource.org/showth...tbound-gateway

I don't have a good fix for this last problem, but the first part looks workable (if ugly). Now I need to figure out how to handle the responses.

[Gary Russell]

1. I assume you are using Spring 3.1; we are aware of the escaping issue and it's being worked on as we speak...

https://jira.springsource.org/browse/INT-2569

2. The fact that your body is in the map under that key implies it is being POSTed as a query parameter rather than the true BODY of the post. When there is a body, it becomes the payload of the message, and the request parameters are available for mapping (on the inbound gateway) using SpEL expressions, with #requestParams as the context variable. You shouldn't need any custom code.

<int:http-inbound-gateway ...>
   <int:header name="accountNumer" expression="#requestParams['accountNumber'] />
   <int:header name="action" expression="#requestParams['action'] />
</int-http:inbound-gateway>

<http:outbound-gateway request-channel="inboundChannel" 
                           url="http://localhost:8080/bps/customer?accountNumber={accountNumber}&action={action}" 
                           http-method="POST"
                           expected-response-type="java.lang.String"
                           reply-timeout="5000" charset="UTF-8">
    <http:uri-variable name="accountNumber" expression="headers['accountNumber']"/>
    <http:uri-variable name="action" expression="headers['action']"/>
</http:outbound-gateway>

(It also solves the escaping problem).

Even if you can't change the behavior of the client (so the body is really the body), you still don't need custom Java code...

<chain input-channel="inboundChannel" output-channel="outboundChannel"
    <header-enricher>
        <header name="content-type" value="application/text"/>
        <header name="accountNumber" expression="payload['accountNumber']"/>
        <header name="action" expression="payload['action']"/>
    </header-enricher>
    <transformer expression="'&lt;notification xmlns=' + payload['&lt;notification xmlns']" />
</chain>

[dheinecke]

Thanks for the help!

Using your suggestion about using uri-variables, I was able to work around the template issue. Here is my definition:

<int:channel id="inChannel"></int:channel>
    <int:channel id="outChannel"></int:channel>
    <int:channel id="replyChannel"></int:channel>
    <bean id="httpReceiver" class="com.mcafee.orion.globaldirector.spring.HttpReceiver"/>

    <http:inbound-gateway request-channel="inChannel" reply-channel="replyChannel" name="/customer"
                          path="/customer" supported-methods="GET,POST">
        <http:header name="accountNumber" expression="#requestParams['accountNumber']"/>
        <http:header name="action" expression="#requestParams['action']"/>
    </http:inbound-gateway>

    <http:outbound-gateway request-channel="outChannel" reply-channel="replyChannel"
                           url="{outBoundGatewayUrl}?accountNumber={accountNumber}&amp;action={action}"
                           http-method="POST"
                           expected-response-type="java.lang.String"
                           reply-timeout="5000" charset="UTF-8">
        <http:uri-variable name="outBoundGatewayUrl" expression="headers['X_REQUEST_URL']"/>
        <http:uri-variable name="accountNumber" expression="headers['accountNumber']"/>
        <http:uri-variable name="action" expression="headers['action']"/>
    </http:outbound-gateway>

    <int:service-activator id="receiver" input-channel="inChannel" output-channel="outChannel" ref="httpReceiver"/>

I do need custom code because even though my example was simple, I need to perform a custom calculation on the request data to determine which DSS to route the request to. In my service activator, I just set the three header values [acct#, action, and url] and it works from there. Getting the result routed back was a little tricky (kept getting exceptions about a missing reply channel), but I figured it out by having my service method return the modified message to the outchannel.

The only ickiness is the handling of the xml request body. I'm pretty sure that the xml is the request body and not a request parameter because I'm using curl to post it to the server.

curl --data createCustomer.xml --insecure --output createCustomerResult.txt --max-time 360 --connect-timeout 360 http://localhost:8080/notification/c...;action=create

The test file just has the following content:

<notification xmlns="http://www.xxx.com/global/rest/schema/notification">
	<context referenceNumber= "0ETU-63546235-FGASDIU-25324534"/>
	<customer companyName="abcTestCustomer1" partnerAccountNumber="9876" locale="en-US"/>
</notification>

[dheinecke]

Just an update in case someone reads this later...

I solved the payload problem. It turns out that if you use the --data parameter with curl, by default it POSTs the file contents with a content-type of application/x-www-form-urlencoded, for which the input gateway parses into name-value pairs.

The solution was to use an additional switch in the curl command line --header Content-Type:text/xml so that the gateway would treat the payload as the xml that it was. Not surprisingly, when the *real* request (not the one from curl) came in from the upstream server, it worked like a charm.

I'm very pleased with the POC that I've put together using SI, and I think it is a lot more robust and handles error conditions better than our http client-based proxy. I even got /fancy/ and used an XPath expression to dig an attribute out of the payload and set it in a header, which wiped out about 50 lines of custom code we had written to do just that very thing. -- very slick!

One question, though: Does SI use async servlet requests to handle inbound gateway requests? I'm using 2.2.0M1. I know that Spring MVC is getting async support in 3.2, and that is the next requirement that I will get pinged about.

Thanks again!