Having difficulty pinpointing problem with Jackson or RestTemplate

[mattupstate]

In my Instagram binding for Spring Social I'm experiencing a strange problem that I can't pin point the culprit of. I feel as if I'm following the 'recipe' for the other bindings but maybe I'm missing something. If you clone the repo and run the unit tests you'll see what I'm talking about. Jackson can't seem to map anything because its reached the end of the stream. In an actual application I get a "stream is closed" IOException. I'd appreciate any guidance or help if anyone could spare the time.

Github:
https://github.com/mattupstate/spring-social

Run the tests:
$ ./gradlew spring-social-instagram:test

[mattupstate]

Application exception:

DEBUG: org.springframework.social.instagram.api.impl.AbstractInstagramOperations - URI: https://api.instagram.com/v1/users/self/?access_token=<hiding token for security>
May 4, 2011 11:15:39 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [appServlet] in context with path [/syrupagram] threw exception [Request processing failed; nested exception is org.springframework.web.client.ResourceAccessException: I/O error: stream is closed; nested exception is java.io.IOException: stream is closed] with root cause
java.io.IOException: stream is closed
	at sun.net.www.http.ChunkedInputStream.ensureOpen(ChunkedInputStream.java:151)
	at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:646)
	at java.io.FilterInputStream.read(FilterInputStream.java:116)
	at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2512)
	at org.codehaus.jackson.impl.ByteSourceBootstrapper.ensureLoaded(ByteSourceBootstrapper.java:340)
	at org.codehaus.jackson.impl.ByteSourceBootstrapper.detectEncoding(ByteSourceBootstrapper.java:116)
	at org.codehaus.jackson.impl.ByteSourceBootstrapper.constructParser(ByteSourceBootstrapper.java:197)
	at org.codehaus.jackson.JsonFactory._createJsonParser(JsonFactory.java:542)
	at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:389)
	at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1455)
	at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.readInternal(MappingJacksonHttpMessageConverter.java:135)
	at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:154)
	at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:74)
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:446)
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:415)
	at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:213)
	at org.springframework.social.instagram.api.impl.AbstractInstagramOperations.get(AbstractInstagramOperations.java:28)
	at org.springframework.social.instagram.api.impl.UserTemplate.getUser(UserTemplate.java:26)
	at org.springframework.social.instagram.connect.InstagramApiAdapter.setConnectionValues(InstagramApiAdapter.java:32)
	at org.springframework.social.instagram.connect.InstagramApiAdapter.setConnectionValues(InstagramApiAdapter.java:11)
	at org.springframework.social.connect.support.AbstractConnection.setValues(AbstractConnection.java:172)
	at org.springframework.social.connect.support.AbstractConnection.initKey(AbstractConnection.java:135)
	at org.springframework.social.connect.support.OAuth2Connection.<init>(OAuth2Connection.java:71)
	at org.springframework.social.connect.support.OAuth2ConnectionFactory.createConnection(OAuth2ConnectionFactory.java:58)
	at org.springframework.social.connect.web.ConnectController.oauth2Callback(ConnectController.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:185)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:101)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMethodAdapter.invokeHandlerMethod(RequestMappingHandlerMethodAdapter.java:501)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMethodAdapter.handleInternal(RequestMappingHandlerMethodAdapter.java:464)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:328)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:95)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:100)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:79)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:35)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:187)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:340)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:175)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.springframework.web.flash.FlashMapFilter.doFilterInternal(FlashMapFilter.java:54)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:562)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:394)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:243)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:188)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:302)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:637)

[Keith Donald]

Matt,
Just curious if the problem happens if you plug in Apache HttpComponents HTTP client?

[mattupstate]

Add it to build.gradle? ...

compile ("org.apache.httpcomponents:httpclient:$httpCompon entsVersion")

[mattupstate]

Here's the unit test exception I'm getting, by the way...

org.springframework.web.client.ResourceAccessException: I/O error: No content to map to Object due to end of input; nested exception is java.io.EOFException: No content to map to Object due to end of input

org.springframework.web.client.ResourceAccessException: I/O error: No content to map to Object due to end of input; nested exception is java.io.EOFException: No content to map to Object due to end of input
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:453)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:415)
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:213)
at org.springframework.social.instagram.api.impl.AbstractInstagramOperations.get(AbstractInstagramOperations.java:23)
at org.springframework.social.instagram.api.impl.LocationTemplate.getLocation(LocationTemplate.java:22)
at org.springframework.social.instagram.LocationTemplateTest.getLocation(LocationTemplateTest.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:51)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:63)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:75)
at $Proxy4.processTestClass(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.remote.internal.MethodInvocationUnmarshallingDispatch.dispatch(MethodInvocationUnmarshallingDispatch.java:48)
at org.gradle.messaging.dispatch.DiscardOnFailureDispatch.dispatch(DiscardOnFailureDispatch.java:31)
at org.gradle.messaging.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:129)
at org.gradle.messaging.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:33)
at org.gradle.messaging.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:69)
at org.gradle.messaging.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:63)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
Caused by: java.io.EOFException: No content to map to Object due to end of input
at org.codehaus.jackson.map.ObjectMapper._initForReading(ObjectMapper.java:2173)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2125)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1455)
at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.readInternal(MappingJacksonHttpMessageConverter.java:135)
at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:154)
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:74)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:446)
... 52 more

[habuma]

The problem originates in InstagramErrorHandler's hasError() method. In that method, you call extractErrorDetailsFromResponse(), which reads through the body stream to see if the body has stuff in it that looks like an error response. If it does, then ultimately the handleError() method will be called...if not, then the normal RestTemplate flow will continue and try to deserialize the body (using Jackson) into a Java object.

The gotcha is that body is an InputStream. When you read the body to determine if there's an error, you advance the stream. When I debug through the LocationTemplateTest.getLocation() test, for instance, the stream's position starts at 0, but is at 175 (the end of the stream) by the time extractErrorDetailsFromResponse() is finished.

If there *is* an error, then handleError() will have a hard time reading it when it calls extractErrorDetailsFromResponse() again. If there isn't an error, then Jackson will pick up where the body stream's current position is...which is at the end of the stream.

If you can trust the HTTP status code to help you determine if this is an error or not, then that's better than reading the body of the response. Just don't override the hasError() method and the default behavior will work for you.

If, however, Instagram could send error responses where the HTTP status code indicates a non-error, then there's little choice but to read the body of the response to determine if you're dealing with an error or not.

I have this same problem with Facebook...which in one case I know of returns an error message with an HTTP 200. I can't trust the HTTP status code, so I have to read the body to decide if it's an error or not. I've not quite got this resolved...when I do, I'll be sure to let you know...in case you still need it for your Instagram module.

[mattupstate]

Ahhh, thanks so much for that explanation. That makes total sense. I'll keep an eye on the Facebook binding for a possible solution.

[habuma]

Okay, I've just pushed change to make this work. But before I walk you through what I did, let me ask...just to be sure...can you trust Instagram's HTTP status codes? If so, please use them to determine if you have an error or not.

In short, have a look at FacebookTemplate (how the request factory is set) and FacebookErrorHandler (how the body is inspected in hasErrors()).

What follows is the longer explanation of how this works...

Arjen has added BufferingClientHttpRequestFactory (and related classes) to Spring 3.1. The responses that come back from requests produced by this factory have a getBody() method that reads the body once into a byte array and then returns a ByteArrayInputStream every time getBody() is called. So, each time you call getBody(), you get a fresh new stream to read from. So as not to depend on Spring 3.1, I've "borrowed" this class and have placed it in the core module's org.springframework.social.support package (along with some other borrowed classes).

FacebookTemplate's constructor and setRequestFactory() method wrap the request factory with a BufferingClientHttpRequestFactory to get this buffering behavior on the response.

Then, in FacebookErrorHandler.hasError(), I call getBody() and inspect just enough of it to decide whether or not there's an error. I also inspect the status code here so that I can avoid reading the body if the status code tells me that there was an error. As I read the body from the input stream, it's just a copy...advancing it won't hurt anything because when I read the body again later, it'll be with a fresh input stream.

The downside to this is that responses will be buffered for every request. BufferingClientHttpRequestFactory has a shouldBuffer() method that I could override to only buffer for calls that I know could be a problem. But by the time I get to the error handler, there's no way of knowing what kind of request was made...so I have to read the body for all responses, which means all responses must be buffered. (I'm still exploring a couple of ideas to limit the buffering to specific request types.)

[mattupstate]

I'd like to hope that Instagram is consistent with its response codes, but I haven't had the chance to verify them all. I'll try and do the rounds to make sure though.

Thanks for the in depth explanation. This, I would assume, is good general knowledge to have when developing a binding for other APIs.