Spring Android and GZIP Compression not being read correctly

[dmfrey]

I am writing an app that should be able to take advantage of a GZIPed response, however, when a GZIP response is detected, the following is thrown:

08-01 11:32:58.620: E/AndroidRuntime(24027): Caused by:
org.codehaus.jackson.JsonParseException: Illegal character ((CTRL-CHAR, code
31)): only regular white space (\r, \n, \t) is allowed between tokens
08-01 11:32:58.620: E/AndroidRuntime(24027): at [Source:
org.apache.http.conn.EofSensorInputStream@40890828 ; line: 1, column: 2]


The 0x31 is the ID1 character in a gziped response.

I am wondering if the JSON parser is being invoked before the GZIP handler. Just a thought.

[Roy Clarkson]

Hi, that shouldn't be the case, but I would like to open a JIRA for this issue so I can look into it further. Can you please provide more details about how you are using ResTemplate, platform, Android device, basically anything that can help me recreate the issue. Thanks for the feedback.

[dmfrey]

Thanks Roy,

I will gather some info from the users reporting the issue and post it to jira.

The app is MythTV for Android and the code is hosted at https://github.com/MythTV-Android/mythtv-for-android.

The actual client implementation is located at https://github.com/MythTV-Android/MythTV-Service-API

The code that invokes the encoding request headers located https://github.com/MythTV-Android/My...perations.java, and the encoding is currently disabled. I will see if I can get a gzip file of the json as well and post it.

[Roy Clarkson]

Ah, very nice. Thanks for the heads up on the Spring usage in the project. I've been a fan of MythTV for a long time.

[dmfrey]

Roy,

That is great to hear.

Spring Android is driving all the interactions between the client and the Myth Services API layer. It has been abstracted out to a separate project since we are building a JUnit test suite against the Myth Services API for regression testing and such. It will actually be implemented in a CI environment as well, but mostly pointing to static JSON resources we have generated from the backend.

I have requested some output of the services from people who are running the .26 version. I am looking forward to both GZIP and ETAG support in the services layer so as to minimize the amount of network resources we are using.

Once I have the output files, I will open a JIRA issue for it.

Dan

[dmfrey]

Created JIRA ANDROID-100 for the issue.

[dmfrey]

I am finally able to reproduce this message myself.

Here is the stack trace that gets produced.

10-22 13:26:09.124: E/AndroidRuntime(9346): FATAL EXCEPTION: IntentService[RecordedDownloadService]
10-22 13:26:09.124: E/AndroidRuntime(9346): org.mythtv.services.api.MythServiceApiRuntimeException: org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]; nested exception is com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]; nested exception is org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]; nested exception is com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.mythtv.services.api.dvr.impl.DvrTemplate.getRecordedList(DvrTemplate.java:342)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.mythtv.service.dvr.RecordedDownloadService.download(RecordedDownloadService.java:153)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.mythtv.service.dvr.RecordedDownloadService.onHandleIntent(RecordedDownloadService.java:118)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at android.os.Handler.dispatchMessage(Handler.java:99)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at android.os.Looper.loop(Looper.java:137)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at android.os.HandlerThread.run(HandlerThread.java:60)
10-22 13:26:09.124: E/AndroidRuntime(9346): Caused by: org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]; nested exception is com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.readJavaType(MappingJackson2HttpMessageConverter.java:170)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.readInternal(MappingJackson2HttpMessageConverter.java:155)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:147)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:76)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:654)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:640)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:483)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:452)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:428)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at org.mythtv.services.api.dvr.impl.DvrTemplate.getRecordedList(DvrTemplate.java:337)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	... 6 more
10-22 13:26:09.124: E/AndroidRuntime(9346): Caused by: com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
10-22 13:26:09.124: E/AndroidRuntime(9346):  at [Source: org.apache.http.conn.EofSensorInputStream@415669b8; line: 1, column: 2]
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1284)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:588)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at com.fasterxml.jackson.core.base.ParserMinimalBase._throwInvalidSpace(ParserMinimalBase.java:534)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd(UTF8StreamJsonParser.java:2326)
10-22 13:26:09.124: E/AndroidRuntime(9346): 	at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8Stream

Is it possible I am just not adding in the correct message converters?

Here is a link to the code:
https://github.com/MythTV-Clients/My...perations.java

I specifically enable the GZIP header on line 117. It is currently commented in git so as not to affect the existing running apps.

[Roy Clarkson]

@dmfrey thanks for the additional information. I'll be working on this issue for inclusion in the 1.0.1 release.

[dmfrey]

Thanks Roy.

Let me know if there is anything I can help out with.

Dan

[Roy Clarkson]

For many reasons, including our Spring One conference, we've had some distractions from this project. But Spring for Android and Spring Mobile will have full attention for the remainder of the year. I'll post in the forums when there are valuable snapshot updates. Pull Requests are always appreciated, of course. If you haven't done so, please submit the contributor agreement [0]. That gives me permission to merge any code from a pull request. Thanks!

[0] https://support.springsource.com/spr...mmitter_signup