Reordering of incoming UDP datagrams

[Miko95]

Hi,

I use a UDP inbound channel adapter to receive incoming UDP datagrams. The UDP protocol doesn't guarantee that the UDP datagrams sent through the network will arrive in order. After a UDP datagram is received, I store it in a queue channel. Is there a possibility to reorder the UDP datagrams in the queue as they arrive?

I've looked at the Resequencer but I understand that it is used to reorder messages that have been splitted using a Splitter. So, I guess it used metadata specific to Spring integration.

Thanks,
Mickael

[Gary Russell]

We added standard headers to the tcp adapters (sequence number) that allow resequencing, but that is only reliable because tcp guarantees packet arrival order. With TCP we might need resequencing if we are using NIO and, although the packets arrive in order, they might be dispatched to the channel out of order. As you say, with UDP, the packets can arrive out of order so assigning sequence numbers on the server side is meaningless.

This means, for UDP, you need to put a sequence number in the payload on the sending side; on the receiving side, you can copy the value from the payload into the sequenceNumber header; you can then use a standard resequencer to resequence the packets.

However, as well as the possibility of packets arriving out of order, UDP doesn't guarantee packet arrival at all; if you have missing packets, a resequencer won't help you because it will wait for the missing packet(s).

The bottom line is, if you want to guarantee arrival and sequence, you should be using TCP, not UDP.

[Miko95]

Thanks for your answer Gary.
So, if I don't need to guarantee arrival but I want to guarantee sequence (even if some UDP datagrams are not present) using a standard Resequencer, is there a way to not wait indefinitely for lost datagrams? some sort of timeout?

By looking at the documentation, I understand this is possible and it's related to the attributes release-partial-sequences, send-partial-result-on-expiry and send-timeout. Am I right? Could you explain me briefly what is the difference between release-partial-sequences and send-partial-result-on-expiry, please?

Thanks

[Gary Russell]

Well, it will work, but it won't be optimal.

First, release partial sequences is for bounded sequences (messages with a sequence size, which would likely not be the case for you).

send-partial-result-on-expiry will work, but all future messages will be delayed.

For example, given this configuration...

	<resequencer input-channel="resequenceChannel" output-channel="outputChannel"
		message-store="messageStore"
		send-partial-result-on-expiry="true"/>

	<beans:bean id="messageStore" class="org.springframework.integration.store.SimpleMessageStore" />

	<beans:bean id="reaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
		<beans:property name="messageGroupStore" ref="messageStore" />
		<beans:property name="timeout" value="2000" />
	</beans:bean>

	<task:scheduled-tasks>	
	  <task:scheduled ref="reaper" method="run" fixed-rate="2000"/>
	</task:scheduled-tasks>

...let's say messages with sequenceNumber=3, then sequenceNumber=2 show up in that order, but #1 is lost.

After the 2 second expiry, messages will be released in the right order; so all seems well. However, the act of expiring the group, causes it to be removed from the message store, this means that when message 4 shows up, even though it's in order, we will again have to wait for the reaper (because now, the resequencer doesn't know that 2 and 3 were already released because it thinks this is a new group).

The other problem is that if, eventually, #1 shows up, it will immediately be released, (after 2 and 3 were already released).

You can probably do what you want to do with a custom release strategy, but it will be difficult to deal with this last edge case (a lost packet eventually showing up).

You might be better off using a custom <service-activator /> to resequence your packets.

This could hold on to (return null for) out of sequence messages and return a List of released packets when a sequence of messages are good. Just put a downstream splitter to break the list back into discrete messages. Your service could then have custom logic to deal with missing packets, and discard stray packets that show up after they were deemed to be missing.

Hope that helps.