Issue accessing Apache James Server using inbound-channel-adapter

[oswulf]

I was trying to access the Apache James Inbox for emails and encountered the following exception:

IMAP DEBUG: IMAPProtocol noop
A7 NOOP
A7 OK NOOP completed.
A8 LIST "" EBTS
* LIST (\HasNoChildren) "." "EBTS"
A8 OK LIST completed.
DEBUG: org.springframework.integration.mail.ImapMailReceiver - opening folder [imap://ebts.incoming@alvinworld.com:*****@alvinworld.com:143/EBTS]
[DEBUG 2012-02-15 17:42:51,965]  opening folder [imap://ebts.incoming@alvinworld.com:*****@alvinworld.com:143/EBTS]
DEBUG: connection available -- size: 1
A9 SELECT EBTS
* FLAGS (\Answered \Deleted \Draft \Flagged \Seen)
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 620175594] UIDs valid
* OK [PERMANENTFLAGS (\Answered \Deleted \Draft \Flagged \Seen \*)] Limited
* OK [UIDNEXT 2] Predicted next UID
A9 OK [READ-WRITE] SELECT completed.
INFO : org.springframework.integration.mail.ImapMailReceiver - attempting to receive mail from folder [EBTS]
[INFO 2012-02-15 17:42:51,981]  attempting to receive mail from folder [EBTS]
DEBUG: org.springframework.integration.mail.ImapMailReceiver - This email server does not support RECENT flag, but it does support USER flags which will be used to prevent duplicates during email fetch.
[DEBUG 2012-02-15 17:42:51,981]  This email server does not support RECENT flag, but it does support USER flags which will be used to prevent duplicates during email fetch.
A10 SEARCH NOT (ANSWERED) NOT (DELETED) NOT (SEEN) NOT (KEYWORD spring-integration-mail-adapter) ALL
A10 BAD SEARCH failed. Illegal arguments.
A11 EXAMINE EBTS
* FLAGS (\Answered \Deleted \Draft \Flagged \Seen)
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 620175594] UIDs valid
* OK [PERMANENTFLAGS (\Answered \Deleted \Draft \Flagged \Seen \*)] Limited
* OK [UIDNEXT 2] Predicted next UID
A11 OK [READ-ONLY] EXAMINE completed.
A12 CLOSE
A12 OK CLOSE completed.
DEBUG: added an Authenticated connection -- size: 1
ERROR: org.springframework.integration.handler.LoggingHandler - org.springframework.integration.MessagingException: failure occurred while polling for mail
	at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:73)
	at org.springframework.integration.endpoint.SourcePollingChannelAdapter.doPoll(SourcePollingChannelAdapter.java:89)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:144)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:207)
	at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
	at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:202)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51)
	at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)
Caused by: javax.mail.MessagingException: A10 BAD SEARCH failed. Illegal arguments.;
  nested exception is:
	com.sun.mail.iap.BadCommandException: A10 BAD SEARCH failed. Illegal arguments.
	at com.sun.mail.imap.IMAPFolder.search(IMAPFolder.java:1547)
	at org.springframework.integration.mail.ImapMailReceiver.searchForNewMessages(ImapMailReceiver.java:124)
	at org.springframework.integration.mail.AbstractMailReceiver.receive(AbstractMailReceiver.java:231)
	at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:59)
	... 18 more
Caused by: com.sun.mail.iap.BadCommandException: A10 BAD SEARCH failed. Illegal arguments.
	at com.sun.mail.iap.Protocol.handleResult(Protocol.java:296)
	at com.sun.mail.imap.protocol.IMAPProtocol.issueSearch(IMAPProtocol.java:1471)
	at com.sun.mail.imap.protocol.IMAPProtocol.search(IMAPProtocol.java:1379)
	at com.sun.mail.imap.protocol.IMAPProtocol.search(IMAPProtocol.java:1367)
	at com.sun.mail.imap.IMAPFolder.search(IMAPFolder.java:1527)
	... 21 more

[ERROR 2012-02-15 17:42:51,981]  org.springframework.integration.MessagingException: failure occurred while polling for mail
	at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:73)
	at org.springframework.integration.endpoint.SourcePollingChannelAdapter.doPoll(SourcePollingChannelAdapter.java:89)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:144)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:207)
	at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:48)
	at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
	at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:202)
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:51)
	at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)
Caused by: javax.mail.MessagingException: A10 BAD SEARCH failed. Illegal arguments.;
  nested exception is:
	com.sun.mail.iap.BadCommandException: A10 BAD SEARCH failed. Illegal arguments.
	at com.sun.mail.imap.IMAPFolder.search(IMAPFolder.java:1547)
	at org.springframework.integration.mail.ImapMailReceiver.searchForNewMessages(ImapMailReceiver.java:124)
	at org.springframework.integration.mail.AbstractMailReceiver.receive(AbstractMailReceiver.java:231)
	at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:59)
	... 18 more
Caused by: com.sun.mail.iap.BadCommandException: A10 BAD SEARCH failed. Illegal arguments.
	at com.sun.mail.iap.Protocol.handleResult(Protocol.java:296)
	at com.sun.mail.imap.protocol.IMAPProtocol.issueSearch(IMAPProtocol.java:1471)
	at com.sun.mail.imap.protocol.IMAPProtocol.search(IMAPProtocol.java:1379)
	at com.sun.mail.imap.protocol.IMAPProtocol.search(IMAPProtocol.java:1367)
	at com.sun.mail.imap.IMAPFolder.search(IMAPFolder.java:1527)
	... 21 more

Currently, i am able to access the same inbox using Thunderbird using IMAP protocol. Below is my spring configuration for your reference:

	<int:channel id="receiveChannel" />
	
	<int-mail:inbound-channel-adapter id="ebts.incoming.adapter"
	      store-uri="imap://ebts.incoming%40alvinworld.com:*******@alvinworld.com:143/EBTS"
	      java-mail-properties="javaMailProperties"
	      channel="receiveChannel"
	      should-delete-messages="false"
	      should-mark-messages-as-read="false"
		  mail-filter-expression="subject matches '(?i).*[EBTS]*'"	      	      
	      auto-startup="true">
	      <int:poller max-messages-per-poll="1" fixed-rate="5000"/>
	</int-mail:inbound-channel-adapter>
	 	
	<util:properties id="javaMailProperties">
	  <prop key="mail.store.protocol">imap</prop>
	  <prop key="mail.debug">true</prop>
	</util:properties>

Thanks in advance.

[oleg.zhurakousky]

Well it is definitely a bug. The only question is it SI bug or James?
The log messages that you posted are very helpful especially this one which basically tells us what is the problem.

[DEBUG 2012-02-15 17:42:51,981]  This email server does not support RECENT flag, but it does support USER flags which will be used to prevent duplicates during email fetch.
A10 SEARCH NOT (ANSWERED) NOT (DELETED) NOT (SEEN) NOT (KEYWORD spring-integration-mail-adapter) ALL
A10 BAD SEARCH failed. Illegal arguments.

So we basically want to get only recent email messages and since James does not support RECENT flag but DOES appear to support USER flags part of the search terms is NOT 'spring-integration-mail-adapter' which is a user flag that would have been set for emails that were retrieved by this adapter, but as you can see James doesn't seem to like it.

I am going to have to test it with James. Any pointers on how to quickly set it up? Also what version of James you are using?

[oleg.zhurakousky]

ALso, could you please raise a JIRA issue https://jira.springsource.org/browse/INT

[oleg.zhurakousky]

Also, I've just downloaded James-3.0-beta. Havn't started it yet but going through the readme it shows

* Now supports 
           IMAP IDLE (RFC 2177, server transmit updates to the client in real time)
           IMAP User Flags

So it would be nice to know what version you have

[oleg.zhurakousky]

Ok, I was able to reproduce it. Let me see what's going on.

[oleg.zhurakousky]

Ok, I actually raised the issue with Apache as I was able to reproduce it outside fo Spring and Spring Integration code.

https://issues.apache.org/jira/browse/JAMES-1375

The thing is that the USER flags is a new feature of James and i was able to successfully set the user flag and see in in the incoming message, however as you can see from the JIRA it fails when user flag is used on the SearchTerm

[oswulf]

Thanks for your prompt reply.

As you had mentioned, the issue might be with Apache James as i am using version 3-beta in order to support IMAP protocol. Anyway, i will raise the issue to Apache James Discussion Group in a short while to confirm this. i might switch to their stable version and use pop3 instead.

The installation and configuration of James-3.0-beta is fairly straightforward (you should be able to get it up and running in 10 minutes):
1) Unzip the archive
2) Goto the /bin and run the run.sh or run.bat script
3) Use the james-cli.sh to create a domain
4) Use the james-cli.sh to add a user

Beside this issue, do you have any idea how i can configure the inbound-channel-adapter to access Microsoft Exchange via OWA (Outlook Web Access)?

Thanks in advance.

Regards,
Alvin

[oswulf]

I am amazed by your prompt action to help to resolve this issue. Thank you very much.

I have switched to POP3 using the same version 3 beta and it seems to be working fine for now. However, i notice that mail-filter-expression do not seem work for POP3. Moreover, i also need your help to connect to Microsoft Exchange Server via OWA as POP3 and IMAP is not allowed in my company. Any advice is appreciated. Thanks again.

	<int-mail:inbound-channel-adapter id="ebts.incoming.adapter"
		store-uri="pop3://ebts.incoming%40alvinworld.com:*******@alvinworld.com:110/INBOX"
		channel="receiveChannel" 
		should-delete-messages="true"
		should-mark-messages-as-read="true"
		mail-filter-expression="subject matches '(?i).*[EBTS]*'"	      	      		
		auto-startup="true"		 
		java-mail-properties="javaMailProperties">
		<!-- Will poll every 20 seconds -->
		<int:poller fixed-rate="20000" />
	</int-mail:inbound-channel-adapter>

[oleg.zhurakousky]

So, the bug is definitely on the Apache James side. I actually just submitted a patch https://issues.apache.org/jira/brows...mment-13209356 So hopefully the will be able to get it in shortly.

As far as the 'mail-filter-expression' it should work. Are you saying it doesn't filter anything? May be the problem is with your regex expression. Try something simpler like:

mail-filter-expression="subject matches '(?i).*foo.*'"

And send an email with subject "hello foobar"

As far as POP3 via OWA, i'd say you need to talk to your admins so they can give you the instructions (e.g., host, port etc.)

[oleg.zhurakousky]

Just to bring it to a close i was able to start James with submitted patch (see https://issues.apache.org/jira/browse/JAMES-1375)
and all works well. So hopefully it will be included shortly