sending large attachments with Spring-WS client

[Sander Hartogensis]

Hi everyone,

With Spring-WS I cannot send large attachments from the client.

I have a Spring-WS web service for receiving files, and with an Axis2 client I have succesfully sent 900mb to it, without even tweaking java's memory-params. But, because we use Spring throughout the application, I want to make it work with a Spring-WS client.

So far, my Spring-WS client crashes with OutOfMemoryError even at 15mb. That is probably because enabling MTOM does not work. With an attachment of 1.2mb TCPMon reveals that my attachment is just base64 encoded within the message.

I had hoped that setting <property name="mtomEnabled" value="true" /> in the marshaller would make it work.

Is this the fault of Spring-WS, or of my code?

WsTest.java:

public class WsTest {
	
	WebServiceTemplate webServiceTemplate;
	
	public void setWebServiceTemplate(WebServiceTemplate webServiceTemplate) {
		this.webServiceTemplate = webServiceTemplate;
	}

	public void sendFile() {
		SendRequest sendRequest = new SendRequest();
		DataSource messageDataSource = new FileDataSource("d:/temp/YaBB_2.2.1.zip");
		sendRequest.setMessage(new DataHandler(messageDataSource));
		this.webServiceTemplate.marshalSendAndReceive("my.first.namespace", sendRequest);
	}
	
	public static void main(String[] args) {
		WsTest wsTest = (WsTest) new ClassPathXmlApplicationContext("applicationContext.xml").getBean("wstest");
		wsTest.sendFile();
	}
}

applicationContext.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="notallowedtoposturls" xmlns:xsi="notallowedtoposturls"
	xsi:schemaLocation="
			notallowedtoposturls">

	<bean id="wstest" class="mtomtest.WsTest">
		<property name="webServiceTemplate">
			<bean class="org.springframework.ws.client.core.WebServiceTemplate">
				<property name="messageFactory">
					<bean id="messageFactory" class="org.springframework.ws.soap.axiom.AxiomSoapMessageFactory"/>
				</property>
				<property name="marshaller" ref="marshaller" />
				<property name="unmarshaller" ref="unmarshaller" />
			</bean>
		</property>
	</bean>

	<bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
		<property name="classesToBeBound">
			<list>
				<value>mtomtest.SendRequest</value>
			</list>
		</property>
		<property name="mtomEnabled" value="true" />
	</bean>

	<bean id="unmarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
		<property name="classesToBeBound">
			<list>
				<value>mtomtest.SendResponse</value>
			</list>
		</property>
	</bean>

</beans>

xsd on which the Jaxb2 classes are based:

<xs:element name="SendRequest">
	<xs:complexType>
		<xs:sequence>
			<!-- some string fields... -->
			<xs:element name="message" type="xs:base64Binary" xmime:expectedContentTypes="*/*" />
		</xs:sequence>
	</xs:complexType>
</xs:element>

Thanks,
Sander Hartogensis

[Arjen Poutsma]

Spring-WS mostly delegates the MTOM functionality to JAXB2, there is very little we do ourself. That said, if you create a JIRA issue, we can take a look at the problem. Please also attach the code you pasted from above, that makes it a lot easier for us to reproduce the problem.

[Sander Hartogensis]

Hi Arjen,

Thanks for the reply. Here is the issue: http://jira.springframework.org/browse/SWS-496

I have created an Eclipse project to demonstrate the problem, and attached it to the issue.

I hope the problem can be resolved, so we don't have to mix SpringWS and Axis in the same project anymore.

Sander

[arcassis]

Hello,

I need to send soap requests and receive responses with large attachments. I've implemented both the WS and the WS client using Spring WS.
Sander, you said that you have successfully sent 900mb with your Axis Client to the WS, but what about the WS, when you want for example to download from the WS a large file, so the response to have a large attachment, did the Spring WS implementation of the WS work well ?

Are there ny news regarding the large files attachments with Spring WS ?

Thanks

[Sander Hartogensis]

Hi Arcassis,

Yes, we have also managed to create a Spring WS endpoint that can send large attachments as a response. We use an Axis2 client for receiving the attachment, because Spring WS clients still handle the attachment in-memory, so they crash on large attachments. The solution involves some nasty not-so-obvious code.

Here are some pointers:

• We used this post as a starting point: http://forum.springsource.org/showthread.php?t=59456
• We use Axiom instead of Saaj. So, that's an AxiomSoapMessageFactory in the config and and an AxiomSoapMessage in the code. I'm not sure if that was necessary for this part of the system (but it definitely was for receiving large attachments from the request).
• Our problem was that our Axis2 ServiceClient would not receive the attachment, even though the reponse was looking fine in TCPMon. If I remember correctly it would not recognize the attachment-reference as an attachment, so we could not get the attachment as a DataSource. Our solution is quite nasty: Instead of getting the attachment via the XML element, we now get it via the attachment-id (study the results in TCPMon if you want to understand this better). So on the server you need to add the attachment with a fixed attachment-id of your own. I will give you the code for the client side:
  

  HashMap<String, MessageContext> messageContexts = serviceClient.getLastOperationContext().getMessageContexts();
  MessageContext messageContext = messageContexts.get("In");
  DataHandler dataHandler = messageContext.attachments.getDataHandler(attachmentId);

• We abstracted away all the ugly bits in specialized classes, so the rest of the code still looks ok. On the server-side we made a ResponseBuilder and a ResponseWriter (which extends TransformerObjectSupport). On the client-side we made AxisSender.sendAndReceiveWithAttachment method, which returns a custom AxisResponse. This AxisResponse has methods to get the attachment as a file or input-stream, and to get the normal elements by name. All in all the end-result is quite readable, but it involved far too much effort for my taste.

Let me know if this solution works for you, or if you succeed with another solution.

Sander

[arcassis]

Thank you Sander for your fast reply and for helping. I will try to reimplement using the Axis2 WS Client.
I'll let you know if I'll find another solution.

Thanks

[arcassis]

Hello Sander,

Could you please post the code for your Spring WS endpoint and Axis 2 client. I've tried implementing them of my own but the endpoint keeps crashing with OutOfMemoryError.

Thank you.

[Sander Hartogensis]

Hi Arcassis,

Sorry for the late reply.

If it's any help, here is my message factory on the endpoint.

<bean id="messageFactory" class="org.springframework.ws.soap.axiom.AxiomSoapMessageFactory">
	<property name="payloadCaching" value="true" />
	<property name="attachmentCaching" value="true" />
	<property name="attachmentCacheDir" value="${directory.incomingFiles}" />
	<property name="soapVersion">
		<util:constant static-field="org.springframework.ws.soap.SoapVersion.SOAP_12" />
	</property>
</bean>

Here is some client code, minus try/catch blocks:

mtomEnableServiceOptions = new Options();
mtomEnableServiceOptions.setProperty(Constants.Configuration.ENABLE_MTOM, Constants.VALUE_TRUE);
mtomEnableServiceOptions.setProperty(Constants.Configuration.CACHE_ATTACHMENTS, Constants.VALUE_TRUE);
mtomEnableServiceOptions.setProperty(Constants.Configuration.FILE_SIZE_THRESHOLD, "1024");
mtomEnableServiceOptions.setTimeOutInMilliSeconds(sendfileMilliSeconds);

EndpointReference endpointReference = EndpointReferenceUtils.createAxis2EndpointReference(webServiceUrl);
mtomEnableServiceOptions.setTo(endpointReference);

ServiceClient serviceClient = new ServiceClient();
serviceClient.setOptions(mtomEnableServiceOptions);

OMElement webServiceResponse = serviceClient.sendReceive(axiomRequest);

I hope this is of some help to you, or any other who might stumble upon this post.

Sander

Hello Sander,

Could you please post the code for your Spring WS endpoint and Axis 2 client. I've tried implementing them of my own but the endpoint keeps crashing with OutOfMemoryError.

Thank you.

[hoffmandirt]

Yes, we have also managed to create a Spring WS endpoint that can send large attachments as a response.

Sander, how large of a file were you able to download from the server to the client?

[mkswapnil]

Hi all,
I want to download large (more than 78 mb) file from server. I check on server side that server sending large attachment to client using soap ui. But in my code i am not getting file.
Please replay to this post ASAP.

Thanks in Advance.