Performance tests

[Arjen Poutsma]

As discussed in another thread I created a little sample app that uses JiBX and Axiom, to see how far I could make Spring-WS go. I profiled it using YourKit, and made some code changes to make it perform even faster.

Normally, Axiom caches all read XML into an internal buffer. However, you can also ask for a non-caching XMLStreamReader, which directly reads from the underlying transport. The drawback is - of course - that you can only read this data once. If you're using plain Axiom (without SWS), using payload caching or not requires a code change (see the JavaDocs here and here).

I didn't like that much, so instead, I added a new property to the AxiomSoapMessageContextFactory: payloadCaching. To quote from the Javadocs:

Setting this property to false will read the contents of the body directly from the TransportRequest. However, [b]when this setting is enabled, the payload can only be read once[b]. This means that any endpoint mappings or interceptors which are based on the message payload (such as the PayloadRootQNameEndpointMapping, the PayloadValidatingInterceptor, or the PayloadLoggingInterceptor) cannot be used. Instead, use an endpoint mapping that does not consume the payload (i.e. the SoapActionEndpointMapping).

By default payloadCaching is true. Pretty self-explanatory, I hope. Since the bulk of the message is bound to be in the message payload, I only made an option for that.

So in your application context, instead of defining:

<bean id="messageContextFactory" class="org.springframework.ws.soap.saaj.SaajSoapMessageContextFactory"/>

you can now define:

<bean id="messageContextFactory" class="org.springframework.ws.soap.axiom.AxiomSoapMessageContextFactory">
        <property name="payloadCaching" value="false"/>
</bean>

to get that extra bit of performance.

Now, I also did some benchmarks to test the performance of this improvement. I deployed the SWS app with JIBX marshalling on a dual 3Ghz Xeon, running Java 5 and the Resin app server. I then used the apache bench (ab) tool to send a 68k soap request 2000 times, using 200 concurrent threads. Obviously, performance tricks like these only make sense with larger messages. Note that, because we use a marshaller, we read the entire payload. Things are very different when you only read a part of it.

I ran three tests, one for each context factory setting. Here are the highlights:

SaajSoapMessageContextFactory with SAAJ 1.3

Requests per second: 34.35 [#/sec] (mean)
Time per request: 5821.728 [ms] (mean)
Time per request: 29.109 [ms] (mean, across all concurrent requests)
Transfer rate: 3.52 [Kbytes/sec] received

AxiomSoapMessageContextFactory with payload caching

Requests per second: 31.02 [#/sec] (mean)
Time per request: 6448.146 [ms] (mean)
Time per request: 32.241 [ms] (mean, across all concurrent requests)
Transfer rate: 3.18 [Kbytes/sec] received

AxiomSoapMessageContextFactory without payload caching
Requests per second: 182.22 [#/sec] (mean)
Time per request: 1097.548 [ms] (mean)
Time per request: 5.488 [ms] (mean, across all concurrent requests)
Transfer rate: 18.68 [Kbytes/sec] received

Quite a drastic increase :-)

In conclusion, here are some best practices regarding message contexts:

• If you require attachment (SwA) support on the server side, use the SaajSoapMessageContextFactory. Axiom does not support adding SwA attachments.
• If you need to read the payload multiple times from the message, use the SaajSoapMessageContextFactory. Axiom is actually slower, though not by far.
• If you only read a part of a big payload using SAX or StAX (multiple times or not), use the AxiomSoapMessageContextFactory, with payload caching. Obviously Axiom does not cache what it does not read.
• If you only want to read the payload once (completely or not), use the AxiomSoapMessageContextFactory, without payload caching.

Since this is only a configuration option, you can play around with it during development time (to have more logging, for instance), and change it when deploying. After all, it does not require any code change.

[res1st]

Hi Arjen,
that's really impressive. I have to see if i can use the AxiomSoapMessageContextFactory in any way.

If i have to transer much data across the net, i.e. big tables with 50000 lines and 1000 per page (of course i have to do paging), is Axiom without cache the right thing?
I mean, i have to put the data into am table widget or table model. Then Axiom is not the right thing, because i have to create a new (string) object for every table entry.
At SAAJ, i simply can put the values into my table model instead of creating a new (string) object.

Are there more efficient ways to transfer so much data?

And what is with the validating interceptor at Axiom (without cache)? Is the interceptor already read-away my data?

Cheers,

Ingo

[Arjen Poutsma]

If i have to transer much data across the net, i.e. big tables with 50000 lines and 1000 per page (of course i have to do paging), is Axiom without cache the right thing?
I mean, i have to put the data into am table widget or table model. Then Axiom is not the right thing, because i have to create a new (string) object for every table entry.
At SAAJ, i simply can put the values into my table model instead of creating a new (string) object.

Well, the API offered for SAAJ and Axiom is the same, from a endpoint point of view. Both provide an implementation of the SoapMessage interface, so if you do something with SAAJ, you can do the same with Axiom.

The only thing is that Axiom only offers direct reads from the stream; it does not offer direct writes. This makes sense, because you might want to add SOAP headers after the endpoint has been written, or you might want to sign the message itself using WS-Security. You can't do that when the payload has already been written to the transport response.

And what is with the validating interceptor at Axiom (without cache)? Is the interceptor already read-away my data?

Well, the name of the class says all: the PayloadValidatingInterceptor. So yes, it reads the payload, which means you can't do it in the endpoint code anymore.

Cheers,

[rwilcom]

So - does the JiBX endpoint support both SAAJ and Axiom?? I thought that they were only going to support Axiom because of marshalling from stream requirements? Did you add the updates so that JiBX with SAAJ would translate to DOM as well?

thanks.
Ron

[Arjen Poutsma]

So - does the JiBX endpoint support both SAAJ and Axiom?? I thought that they were only going to support Axiom because of marshalling from stream requirements? Did you add the updates so that JiBX with SAAJ would translate to DOM as well?

Yes, the JibxMarshaller supports both SAAJ and DOM. That's the idea behind the whole WebServiceMessage and SoapMessage abstraction: it's doesn't matter what underlying API is used: it should work with everything. The same idea goes for all the Marshallers: all of them work with DOM, SAX, StAX and I/O streams.

That said, it isn't necessary fast. For instance, when a JibxMarshaller is asked to unmarshal a DOM document (like contained in a SAAJ message), it is first converted to an in-memory stream, which is then read. This is due to the fact that JiBX API only supports reading from I/O streams, or StAX stream readers. So the whole DOM to stream conversion is done by Spring-WS, to make sure that it works. And that's where Axiom comes in. Axiom uses StAX, JiBX can use StAX, so there's a match there, and it's pretty fast, especially if you disable the caching.

This functionality wasn't always there, but has been added about a month ago (see http://opensource.atlassian.com/proj.../browse/SWS-36)

[Daniel Rijkhof]

As discussed in another thread I created a little sample app that uses JiBX and Axiom, to see how far I could make Spring-WS go.

This tread doesn't seem to exist anymore. I want to make a simple webservice that uses Spring-WS with JiBX and Axiom. Your sample app would be much appreciated.