UTF-8 and Latin Characters

[gsmiro]

Good folks. I've been playing around with roo for some time and I'm facing an issue with encodings. I'm brazilian so I'd like to enable my application to handle latin characters.

Now, I found that, after simply creating a controller for one of my entities, if I enter a character like 'ó', I get the string 'ó' in the controller method.

I'm using MySQl with utf-8 and haven't touched the code generated by Roo. I noticed the JSP has Content-Type Meta tag set to UTF-8, so seems like everything should be working.

My only clue is the following site:http://wiki.apache.org/tomcat/FAQ/CharacterEncoding

which states that Tomcat handles GET and POST as ISO-8859-1.

I'd like to know if this is something Roo doesn't cover-which I believe is not true- or if I'm missing something-and please let me know what.

Regards,
Gabriel Miró

[MiB]

Did you try the suggestions in the section What can you recommend to just make everything work? (How to use UTF-8 everywhere). at the URL you gave?

I have similar needs as I make swedish sites. So far, I haven't had the problems you are having, except for some message string in properties-files.

[gsmiro]

I noticed that there's an org.springframework.web.filter.CharacterEncodingFi lter(web.xml) set to UTF-8 and the jsp's specify UTF-8 as the encoding, so everything should be working.

Yet, whenever I enter latin characters I get them jammed at the controller.

[gsmiro]

Ok. I did some testing and I found out that the issue only happens if I run the application in STS's (Version: 2.3.0.RELEASE Build Id: 200912171331) tc Server (v6.0).

Running it through maven I cannot see the issue.

[serhii]

Check the filter mappings in your web.xml.

The filter mapping for the CharacterEncodingFilter must be the first, as explained in ROO-1731

[gsmiro]

So this is my web.xml filter mappings

<filter-mapping>
<filter-name>Spring OpenEntityManagerInViewFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
<filter-name>etagFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>CharacterEncodingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>httpMethodFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>UrlRewriteFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>


I tried to put character mapping on top:

<filter-mapping>
<filter-name>CharacterEncodingFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>


<filter-mapping>
<filter-name>Spring OpenEntityManagerInViewFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
<filter-name>etagFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>httpMethodFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

<filter-mapping>
<filter-name>UrlRewriteFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

and it didn't fix it. Any ideas?

[MiB]

gsmiro, what are the relevant Spring projects involved? Is it just MVC or are you using Spring Web Flow as well or something like that?

[gsmiro]

I'm using STS to run a Roo project. As I haven't added security to the project, I guess it's Spring MVC as used by Roo.

I was using STS 2.3 and Roo 1.1.0RC4. So this weekend I updated STS to 2.6 and Roo to 1.1.2 and created a new project from scratch. I noticed that the the CharacterEncodingFilter was set first when roo generated web.xml, according to what was mentioned in ROO-1731 and related issues, but I'm still facing issues.

I noticed that tomcat uses ISO-8859-1 to URIEncode by default. I tried to change it to UTF-8 in the HTTP connector but apparently it didn't have any effect - I'm still testing this.

Also, I'm using STS tcServer to test.
Last note is that I set a breakpoint in the CharacterEncodingFilter and inspect the request and the string was already wrong there, so it makes me thinks it isn't just the filter.

Here's log.roo:

// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log opened at 2011-03-27 21:00:14
project --topLevelPackage com.gm.test --projectName Blah --java 6
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log closed at 2011-03-27 21:00:21
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log opened at 2011-03-27 21:00:48
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log closed at 2011-03-27 21:03:13
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log opened at 2011-03-27 21:03:32
// [failed] hint
exit
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log closed at 2011-03-27 21:05:09
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log opened at 2011-03-27 21:21:19
hint
persistence setup --database HYPERSONIC_IN_MEMORY --provider HIBERNATE
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log closed at 2011-03-27 21:39:01
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log opened at 2011-03-27 22:28:17
hint
entity --class ~.model.Item
hint
field boolean --fieldName name --notNull
hint
hint logging
logging setup --level DEBUG
hint
controller scaffold --class ~.web.ItemController --entity ~.model.Item
// [failed] entity --class ~.model.Item
perform eclipse
// Spring Roo 1.1.2.RELEASE [rev fbc33bb] log closed at 2011-03-28 00:38: