Character Set in CF for German Umlaut Characters

What character set should I be using when trying to display German Umlaut characters (ä ö ü ß)?  I'm using utf-8 throughout the site.  I'm using CF 9 with SQL 2012.  

Sometimes if I place the umlaut character inside a cfinput or cftextarea, etc. all I get is the ä instead of ä.  I have the umlaut characters as values in SQL DB but that info not displaying properly either.  Also...within some ajax areas of the site, the umlaut characters are replaced with a box.  

I've seen some posts throughout the web referring to using ISO-8859-1.  Not sure if this is a fix all and if it's going to affect and mess up the rest of the site.

Ideas on how to resolve is greatly appreciated at this point.  Thank you.
Lee R Liddick JrReporting AnalystAsked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
Advertise Here
 
_agx_Connect With a Mentor Commented:
Do you have a small example that reproduces the issue? I tried a test page w/CF9, FF and SQL Server 2005 and unfortunately .. couldn't reproduce the issue you described. Even without setting the charset explicitly.

I'm not quite clear on how the problem is manifesting itself. ie Are the characters NOT being saved correctly to the db =OR= are the characters saved correctly in the db, but when displayed in HTML form fields they're incorrect?

Just in the general area of encoding problems, other settings you might check:

- Do your Application files use SetEncoding(...) at all?
- If the problem is characters not being stored correctly in the db, check your DSN settings. Does it enable the String Format setting?


Sometimes if I place the umlaut character inside a cfinput or cftextarea, etc. all I get is the ä instead of ä.

Place how? Do you mean you copy+paste, and the pasted values are incorrect? Honestly I don't use cfinput/cftextarea much, but created a test form (see below) and tested with: cfinput, cftextarea, cftextarea richtext=true" and didn't observe any problems. (That's without changing the page encoding). I know you mentioned cfinput's too, but ... could it be a problem with the underlying FCKEditor?

http://ckeditor.com/forums/Support/FCK-UTF-8-encoding-FCKeditor-not-working-special-charac
http://dev.ckeditor.com/ticket/490


---------------------

SQL: (Default Collation: SQL_Latin1_General_CP1_CI_AS)

CREATE TABLE testTable
(
id int identity
, qn_nm nvarchar(255) NOT NULL
, createdDate datetime default(getDate())
)

Test characters:
www.columbia.edu/~fdc/utf8/

Test Form:

<cfset dsn = "MyDataSourceName">

<cfparam name="FORM.idList" default="" />

<cfif structKeyExists(FORM, "newValue")>
      
      <!--- TEST: Add new record --->
      <cfif len(trim(FORM.newValue))>
            <cfquery name="qTest" datasource="#dsn#">
                  INSERT INTO TestTable( qn_nm )
                  VALUES (
                        <cfqueryparam value="#FORM.newValue#" cfsqltype="cf_sql_varchar" />
                  )
            </cfquery>
      </cfif>
      
      <!--- TEST: Update existing records --->
      <cfloop list="#FORM.idList#" index="VARIABLES.id">
            <cfquery name="qTest" datasource="#dsn#">
                  UPDATE       TestTable
                  SET       qn_nm = <cfqueryparam value="#FORM['updateValue_'& VARIABLES.ID]#" cfsqltype="cf_sql_varchar" />
                  WHERE   ID = <cfqueryparam value="#VARIABLES.ID#" cfsqltype="cf_sql_integer" />
            </cfquery>
      </cfloop>
</cfif>

<!--- get existing records --->
<cfquery name="qTest" datasource="#dsn#">
      SELECT ID, CreatedDate, qn_nm
      FROM   testTable
      ORDER BY ID DESC
</cfquery>

<!--- TEST: Edit values --->
<cfform method="post">
      New: <cfinput type="text" name="newValue"/> <br/>
      Update: <br/>
      <cfoutput query="qTest">
            [#currentRow#] <input type="hidden" name="idList" value="#ID#" />
            <cfinput type="text" name="updateValue_#ID#" value="#qTest.qn_nm#" size="50" /><br />
      </cfoutput>
      <input type="submit" />
</cfform>

<!--- TEST: DISPLAY as text --->
<cfquery name="qTest" datasource="#dsn#">
      SELECT ID, CreatedDate, qn_nm
      FROM   testTable
      ORDER BY ID DESC
</cfquery>
<cfoutput query="qTest">
      #ID# | #qn_nm#<br/>
</cfoutput>
0
 
GaryConnect With a Mentor Commented:
Use the entity code not the HTML code e.g.
Ä = &#196;
0
 
Lee R Liddick JrReporting AnalystAuthor Commented:
No go cathal...didn't work.  Still get the entity code showing on the page.  Actually I got a CF error first because it was confused with the single #.
0
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Access It Now
 
GaryConnect With a Mentor Commented:
Are you sure your page is saved as UTF-8 and/or the server is sending an UTF-8 header?
Can you attach your current page
Simple page showing there is no problem with either character (sometimes the HTML code can be iffy depending on other things)
http://jsfiddle.net/GaryC123/uNY3j/1/
0
 
_agx_Connect With a Mentor Commented:
It sounds like you're talking about several different scenarios, so the exact cause and solutions may vary.

Can you provide a small repro case that demonstrates the issue w/cfinput (and also an "ajax" example)

Specifically:
Edit: What are your DSN settings?
1) Database column type? Are the values stored as NVARCHAR?
2) Example of actual values stored , ie" xxxxäxxx,  xxxöxxx, ... etc
3) Query used to retrieve db value
4) Code used to output the query value
0
 
Lee R Liddick JrReporting AnalystAuthor Commented:
1) The database column type is set as qn_nm (nvarchar(255),not null)
2) Example, Möchten Sie an der Umfrage teilnehmen?
3)  select * from rfd_svy_dt x, rfd_qn y where x.qn_id = y.qn_id and x.svy_id = #arguments.svy_id# and svy_qn_seq_n = #arguments.qnum#
4) #qn_nm1# within a cfloop

Pages are set for <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
0
 
Lee R Liddick JrReporting AnalystAuthor Commented:
I've now tried to change the DB collation, replaced all utf-8 references to UTF-8, and made sure every page reference showed charset='UTF-8'.  I have yet to find anything that works.  Still searching...deadline approaching March 27.
0
 
Lee R Liddick JrReporting AnalystAuthor Commented:
We ended up resolving this within our code and database values.  We found an issue with our translation code that was not processing on the cf page correctly.  Once we corrected that, the umlauts were displaying properly within the pages.  

I awarded points to all who took the time to respond.  Thank you!
0
 
_agx_Commented:
> translation code

Glad to hear you were able to solve it!  If you get a chance - do you know what code/encoding specifically caused the problem?  (I was not clear on that) Might help the next guy that runs into it.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.