How to write a concurrent web service client

[scranthdaddy]

Hi, I have a requirement to parse a file that contains about 5,000 records and for each record, call a web service. Since each request can take up to 1 minute to get a response back, I would like to make the service calls concurrently. I am honestly not sure where to start and am looking for best practice suggestions.

Here is how I have defined my spring beans (this actually works):

    <bean id="xboxService" class="com.scranthdaddy.xbox.service.XboxServiceImpl">
        <property name="webServiceTemplate" ref="webServiceTemplate"/>
    </bean>

    <bean id="marshaller" class="org.springframework.oxm.xmlbeans.XmlBeansMarshaller"/>

    <bean id="webServiceTemplate" class="org.springframework.ws.client.core.WebServiceTemplate">
        <property name="defaultUri" value="http://xbox2.scranthdaddy.com"/>

        <property name="marshaller" ref="marshaller"/>

        <property name="unmarshaller" ref="marshaller"/>
    </bean>

Thanks!

[scranthdaddy]

I guess I will answer my own question Hopefully it helps others attempting to do the same thing.

Here is how I have now defined my spring beans (this actually works):

    <bean id="xboxService" class="com.scranthdaddy.xbox.service.XboxServiceImpl">
        <property name="webServiceTemplate" ref="webServiceTemplate"/>
    </bean>

    <bean id="messageSender" class="org.springframework.ws.transport.http.CommonsHttpMessageSender">
        <property name="connectionTimeout" value="15000"/>

        <property name="readTimeout" value="60000"/>

        <property name="maxTotalConnections" value="20"/>

        <property name="maxConnectionsPerHost">
            <props>
                <prop key="*">20</prop>
            </props>
        </property>
    </bean>

    <bean id="marshaller" class="org.springframework.oxm.xmlbeans.XmlBeansMarshaller"/>

    <bean id="webServiceTemplate" class="org.springframework.ws.client.core.WebServiceTemplate">
        <property name="defaultUri" value="http://xbox2.scranthdaddy.com"/>

        <property name="messageSender" ref="messageSender"/>

        <property name="marshaller" ref="marshaller"/>

        <property name="unmarshaller" ref="marshaller"/>
    </bean>

My first question: Is calling the xboxService thread-safe and can I call the xboxService concurrently? The answer is yes, because by default the CommonsHttpMessageSender uses HttpClient's MultiThreadedHttpConnectionManager. Just be sure to set the messageSender properties per your requirements.

My next question: How I do call the xboxService concurrently? The answer was (for my use case) to use some java util concurrent classes:

Main.java

package com.scranthdaddy;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
    public static void main(String[] args) throws Exception {
        Main main = new Main();
        main.runBatchThreads();
    }

    private void runBatchThreads() {
        // initialize list of WebServiceTask objects
        List<WebServiceTask> webServiceTasks = new ArrayList<WebServiceTask>();

        for (int i = 0; i < 5000; i++) {
            WebServiceTask webServiceTask = new WebServiceTask();

            webServiceTasks.add(webServiceTask);
        }

        System.out.println("Starting threads");

        // create ExecutorService to manage threads
        ExecutorService executorService = Executors.newFixedThreadPool(20);

        for (WebServiceTask webServiceTask : webServiceTasks) {
            // start thread
            executorService.execute(webServiceTask);
        }

        // shutdown worker threads when complete
        executorService.shutdown();

        System.out.println("Threads started, main ended");
    }
}

WebServiceTask.java

package com.scranthdaddy;

public class WebServiceTask implements Runnable {
    public void run() {
        // call xboxService ...        
    }
}

This is really cool because if you use jconsole, you can see the 20 threads started up. And then when then threads are all done they get shutdown. Nice job by the Sun/Oracle developers!

[nkunkov]

Hi, first - thank you for posting your solution!
I'm to build something similar but without Spring.
I will have a java client and a java webservice running remotely (I'll be using Axis2). I will also iterate over a large file and will fire asynchronous web service calls.
I see that you are limiting the calls on the client to 20, right? How did you come up with this number? I'm wondering what is a reasonable number of concurrent web service calls and how to determine it? What happens if you fire too many?
Would appreciate your comments.
Thanks
NK

[scranthdaddy]

I see that you are limiting the calls on the client to 20, right? How did you come up with this number? I'm wondering what is a reasonable number of concurrent web service calls and how to determine it? What happens if you fire too many?

I'm not a threading expert but I understand that if you fire too many threads you could take your server down. I just chose 20 because it was just enough to get the batch completed in a reasonable amount of time. I bought the book Java Concurrency in Practice and there is a formula for determining how many threads to use in your thread pool, but I didn't go by that. Trial and error worked for me, otherwise try posting your question on forums.oracle.com under Java APIs --> Concurrency. Hope that helps!

[priyatham2]

I've used CommonsHttpMessageSender as a messageSender. But got the below error in a multi threaded environment.

SimpleHttpConnectionManager being used incorrectly. Be sure that HttpMethod.releaseConnection() is always called and that only one thread and/or method is using this connection manager at a time.

I see that the constructor of CommonsHttpMessageSender creates an instance of MultiThreadedConn manager but wondering why the SimpleHttpConn manager was used in my case.

- Priyatham Sundar

[priyatham2]

Ok. I think I found the issue. My httpSender (CommonsHttpMessageSender) uses a httpClient. This was done to set the proxy host and port. While declaring the httpClient bean (apache commons HttpClient), the httpConnectionManager was not set. (Should have been set to MultiThreadedHttpConnectionManager). Without this , the httpConnectionManager in httpClient defaults to SimpleHttpConnectionManager. This was the issue.

I've corrected the config to set MultiThreadedHttpConnectionManager as the httpConnectionManager on the httpClient.