Solved

Character Set in CF for German Umlaut Characters

Posted on 2014-01-04
9
1,055 Views
Last Modified: 2014-03-19
What character set should I be using when trying to display German Umlaut characters (ä ö ü ß)?  I'm using utf-8 throughout the site.  I'm using CF 9 with SQL 2012.  

Sometimes if I place the umlaut character inside a cfinput or cftextarea, etc. all I get is the ä instead of ä.  I have the umlaut characters as values in SQL DB but that info not displaying properly either.  Also...within some ajax areas of the site, the umlaut characters are replaced with a box.  

I've seen some posts throughout the web referring to using ISO-8859-1.  Not sure if this is a fix all and if it's going to affect and mess up the rest of the site.

Ideas on how to resolve is greatly appreciated at this point.  Thank you.
0
Comment
Question by:Lee R Liddick Jr
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
9 Comments
 
LVL 58

Assisted Solution

by:Gary
Gary earned 100 total points
ID: 39756780
Use the entity code not the HTML code e.g.
Ä = Ä
0
 

Author Comment

by:Lee R Liddick Jr
ID: 39756794
No go cathal...didn't work.  Still get the entity code showing on the page.  Actually I got a CF error first because it was confused with the single #.
0
 
LVL 58

Assisted Solution

by:Gary
Gary earned 100 total points
ID: 39756804
Are you sure your page is saved as UTF-8 and/or the server is sending an UTF-8 header?
Can you attach your current page
Simple page showing there is no problem with either character (sometimes the HTML code can be iffy depending on other things)
http://jsfiddle.net/GaryC123/uNY3j/1/
0
More Than Just A Video Library

Train for your certification. Learn the latest DevOps tools. Grow your skillset to do better work.

At Linux Academy, we release new training modules every week so you'll always be up to date on the latest tech.

 
LVL 52

Assisted Solution

by:_agx_
_agx_ earned 400 total points
ID: 39758094
It sounds like you're talking about several different scenarios, so the exact cause and solutions may vary.

Can you provide a small repro case that demonstrates the issue w/cfinput (and also an "ajax" example)

Specifically:
Edit: What are your DSN settings?
1) Database column type? Are the values stored as NVARCHAR?
2) Example of actual values stored , ie" xxxxäxxx,  xxxöxxx, ... etc
3) Query used to retrieve db value
4) Code used to output the query value
0
 

Author Comment

by:Lee R Liddick Jr
ID: 39900280
1) The database column type is set as qn_nm (nvarchar(255),not null)
2) Example, Möchten Sie an der Umfrage teilnehmen?
3)  select * from rfd_svy_dt x, rfd_qn y where x.qn_id = y.qn_id and x.svy_id = #arguments.svy_id# and svy_qn_seq_n = #arguments.qnum#
4) #qn_nm1# within a cfloop

Pages are set for <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
0
 

Author Comment

by:Lee R Liddick Jr
ID: 39929016
I've now tried to change the DB collation, replaced all utf-8 references to UTF-8, and made sure every page reference showed charset='UTF-8'.  I have yet to find anything that works.  Still searching...deadline approaching March 27.
0
 
LVL 52

Accepted Solution

by:
_agx_ earned 400 total points
ID: 39932103
Do you have a small example that reproduces the issue? I tried a test page w/CF9, FF and SQL Server 2005 and unfortunately .. couldn't reproduce the issue you described. Even without setting the charset explicitly.

I'm not quite clear on how the problem is manifesting itself. ie Are the characters NOT being saved correctly to the db =OR= are the characters saved correctly in the db, but when displayed in HTML form fields they're incorrect?

Just in the general area of encoding problems, other settings you might check:

- Do your Application files use SetEncoding(...) at all?
- If the problem is characters not being stored correctly in the db, check your DSN settings. Does it enable the String Format setting?


Sometimes if I place the umlaut character inside a cfinput or cftextarea, etc. all I get is the &auml; instead of ä.

Place how? Do you mean you copy+paste, and the pasted values are incorrect? Honestly I don't use cfinput/cftextarea much, but created a test form (see below) and tested with: cfinput, cftextarea, cftextarea richtext=true" and didn't observe any problems. (That's without changing the page encoding). I know you mentioned cfinput's too, but ... could it be a problem with the underlying FCKEditor?

http://ckeditor.com/forums/Support/FCK-UTF-8-encoding-FCKeditor-not-working-special-charac
http://dev.ckeditor.com/ticket/490


---------------------

SQL: (Default Collation: SQL_Latin1_General_CP1_CI_AS)

CREATE TABLE testTable
(
id int identity
, qn_nm nvarchar(255) NOT NULL
, createdDate datetime default(getDate())
)

Test characters:
www.columbia.edu/~fdc/utf8/

Test Form:

<cfset dsn = "MyDataSourceName">

<cfparam name="FORM.idList" default="" />

<cfif structKeyExists(FORM, "newValue")>
      
      <!--- TEST: Add new record --->
      <cfif len(trim(FORM.newValue))>
            <cfquery name="qTest" datasource="#dsn#">
                  INSERT INTO TestTable( qn_nm )
                  VALUES (
                        <cfqueryparam value="#FORM.newValue#" cfsqltype="cf_sql_varchar" />
                  )
            </cfquery>
      </cfif>
      
      <!--- TEST: Update existing records --->
      <cfloop list="#FORM.idList#" index="VARIABLES.id">
            <cfquery name="qTest" datasource="#dsn#">
                  UPDATE       TestTable
                  SET       qn_nm = <cfqueryparam value="#FORM['updateValue_'& VARIABLES.ID]#" cfsqltype="cf_sql_varchar" />
                  WHERE   ID = <cfqueryparam value="#VARIABLES.ID#" cfsqltype="cf_sql_integer" />
            </cfquery>
      </cfloop>
</cfif>

<!--- get existing records --->
<cfquery name="qTest" datasource="#dsn#">
      SELECT ID, CreatedDate, qn_nm
      FROM   testTable
      ORDER BY ID DESC
</cfquery>

<!--- TEST: Edit values --->
<cfform method="post">
      New: <cfinput type="text" name="newValue"/> <br/>
      Update: <br/>
      <cfoutput query="qTest">
            [#currentRow#] <input type="hidden" name="idList" value="#ID#" />
            <cfinput type="text" name="updateValue_#ID#" value="#qTest.qn_nm#" size="50" /><br />
      </cfoutput>
      <input type="submit" />
</cfform>

<!--- TEST: DISPLAY as text --->
<cfquery name="qTest" datasource="#dsn#">
      SELECT ID, CreatedDate, qn_nm
      FROM   testTable
      ORDER BY ID DESC
</cfquery>
<cfoutput query="qTest">
      #ID# | #qn_nm#<br/>
</cfoutput>
0
 

Author Closing Comment

by:Lee R Liddick Jr
ID: 39938817
We ended up resolving this within our code and database values.  We found an issue with our translation code that was not processing on the cf page correctly.  Once we corrected that, the umlauts were displaying properly within the pages.  

I awarded points to all who took the time to respond.  Thank you!
0
 
LVL 52

Expert Comment

by:_agx_
ID: 39939714
> translation code

Glad to hear you were able to solve it!  If you get a chance - do you know what code/encoding specifically caused the problem?  (I was not clear on that) Might help the next guy that runs into it.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hi, I will be creating today a basic tutorial on how we can create a Mail Custom Function and use it where ever we want. The main advantage about creating a custom function is that we can accommodate a range of arguments to pass to the Function and …
This is an updated version of a post made on my blog over 3 years ago. It is unfortunately, still very relevant as we continue to see both SQLi (SQL injection) and XSS (cross site scripting) attacks hitting some of the most recognizable website and …
Monitoring a network: how to monitor network services and why? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the philosophy behind service monitoring and why a handshake validation is critical in network monitoring. Software utilized …
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question