Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Another facebox issue: international characters not displayed

Posted on 2008-10-23
11
Medium Priority
?
1,126 Views
Last Modified: 2012-08-13
Hello again experts

Thanks to sh0e's patience I got my previous issue sorted and I'm doing even more with facebox and everyone loves it. However I'm now going international and have some new files that need to be categorised in the database. The categorisation will all be in English but the file name and its associated label must remain in French.

I'm seeing an issue where all accented (diacritic) characters are displayed as ? or a box (depending on whether it's firefox or IE) when a facebox is opened. Now, I'm using ISO-8859-1 as my character set, as when I specify UTF-8 I get boxes everywhere there's an accent, not just in the facebox page. I'm wondering if my elderly version of ultraedit is to blame.

I'll post the code of the facebox page below, as I'm afraid I'm on a corporate intranet and can't really put demo a page fully online (yet). This page displays fine in its own browser window. Jquery is being properly loaded in the background window. I've removed the serverside code as I know it's working fine. The variables are encapsulated with $.

Also, I say it's a facebox problem, but a few moments ago I tried doing the same thing quickly using thickbox -- with the same results. So that's got me wondering where on earth I'm going wrong?

Hopefully someone can help. 500 points, as ever :-D


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://
www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="fr">
<head>
<meta http-equiv="content-type" content="text/
html;charset=ISO-8859-1" />
 
<style>
#filearticle {padding:0; margin:0}
#filearticle form {margin:0; padding: 0}
h1.articletitle {font-size: 10pt; color: #FFFFFF; background: #0077bb;
padding: 2px; margin: 0 }
#articlebody {padding: 5px}
#articlebody p {margin: 5px 0 5px 0}
#articlebody ul {margin: 5px 40px 5px 30px; list-style-type: square}
dl{
        width: 100%;
        padding: 0;
        background:#FFFFFF;
        margin: 0;
        text-align: left;
        clear: both;
        background: none;
 
}
 
dl dt{
        float: left;
        width: 25%;
        padding: .2em 0% 0.5em 0%;
        margin: 0 ;
        font-weight: bold;
        clear: left;
        font-size: 8pt;
 
}
 
dl dd{
        width: 70%;
        padding: .2em 0% 0.5em 0;
        float: left;
        margin: 0 ;
        font-size: 8pt;
}
 
input {border:1px solid #808080; background: #EEEEEE; font-size: 8pt;
font-family: Tahoma}
.quickalert1 {margin: 0;background: #0077bb; color: #FFFFFF; text-
align: center; font-size: 10pt; font-weight: normal; padding: 10px;
width: 60%;position: absolute; top: 35px ; left: 20%;border-top:1px
solid #99ccff; border-left:1px solid #99ccff;border-right:1px solid
#003366;border-bottom:1px solid #003366;}
#filelabelformsubmitted {display:none;}
</style>
<script type="text/javascript" src="js/jquery.form.min.js"></script>
 
<script type="text/javascript">
 
    $('#facebox #filelabelform').ajaxForm({
        target:       '#facebox #filelabelformsubmitted',
        beforeSubmit:  function(){
                       var filecategory1v = $
('select[@name=filecategory1]').fieldValue();
                       var filecategory2v = $
('select[@name=filecategory2]').fieldValue();
                       var filelanguagev  = $
('select[@name=filelanguage]').fieldValue();
                       var filepositionv  = $
('select[@name=fileposition]').fieldValue();
 
                       if (!filecategory1v[0] || !filecategory2v[0]
|| !filelanguagev[0] || !filepositionv[0]) {
                       $("#filelabelformsubmitted").html('You must
select from <b>ALL</b> dropdowns.').fadeIn('slow').animate({opacity:
1.0}, 2000).fadeOut('slow');
                       return false;
                       }
                       return true;
 
                       },
        success:       function(){
                       $('#facebox
#filelabelformsubmitted').fadeIn('slow').animate({opacity: 1.0},
2000).fadeOut('slow');
                       $('#facebox').animate({opacity: 1.0},
10000).fadeOut('slow');
                       $('#facebox_overlay').remove();
                       $("#filecat_$f
$").removeClass("categorylink").addClass("categorylinkdone");
                       }
    });
    </script>
 
</head>
<body>
 
<div id="filelabelformsubmitted" class="quickalert1"></div>
<div id="filearticle">
<h1 class="facebox_title">Categorise File</h1>
<div id="articlebody">
 
<form id="filelabelform" method="post" action="filelabelscript.dpe">
 
<dl>
<!--NOTE : when filename is displayed below it has ? or square for
accented characters depending on browser.-->
<dt>File name:</dt><dd>$filename$</dd>
<dt>File entity:</dt><dd>$entity$</dd>
<input type="hidden" name="filename" value="$filename$">
<input type="hidden" name="entity" value="$entity$">
<dt>File label:</dt><dd><input type="text" name="filelabel" size="40"
value="$filelabel1$" id="filelabel"></dd>
<dt>File Category1:</dt><dd><select size="1" name="filecategory1"
id="filecategory1">
 
                               <option value="">Select...</option>
 
                               <option value="$id$" selected>$category
$</option>
 
                              </select></dd>
 
<dt>File Category2:</dt><dd><select size="1" name="filecategory2"
id="filecategory2">
 
                               <option value="">Select...</option>
 
                               <option value="$id$" selected>$category
$</option>
 
                              </select></dd>
 
<dt>File Language:</dt><dd><select size="1" name="filelanguage"
id="filelanguage">
 
                               <option value="">Select...</option>
 
                               <option value="$id$" selected>$language
$</option>
 
                              </select></dd>
<dt>File Positioning:</dt><dd><select size="1" name="fileposition"
id="fileposition">
 
                               <option value="">Select...</option>
 
                               <option value="$id$" selected>$position
$</option>
 
                              </select>
 
</dd>
<dt></dt><dd><input type="submit" value="submit" id="btnSubmit"></dd>
</dl>
</form>
 
</div>
</div>
</body>
</html>

Open in new window

0
Comment
Question by:Dave6969
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 2
11 Comments
 
LVL 75

Expert Comment

by:Michel Plungjan
ID: 22795371
If the characters are saved as DOS iso-8859-1 then you need iso-8859-1
If they are saved as UTF8, you need UTF8
If you use entities (&egrave; &euml;) then it does not matter
0
 

Author Comment

by:Dave6969
ID: 22796634
Actually since no one was answering I got to grips with it a bit. I've done some replacing server side and I can get it to display fine.

HOWEVER, there is a form field in the page for the file label. It's going to be mostly english speakers categorising this as first. If you type accented characters into the field (as a french speaker would do) there's no problem submitting, but if you copy and paste (as an english speaker will do) from the displayed filename (with the HTML entities in it) then a load of gobbledegook gets entered into the database where the HTML entities used to be.

I tried writing some jquery code to replace the entities on-the-fly, but of course I hit the same problem with the characters not displaying in the page. Very frustrating!
0
 
LVL 16

Expert Comment

by:sh0e
ID: 22796825
It's already done, but usually the solution seems to be to force encoding on the server side.

It sounds like you are trying to sanitize entities on the client side.  I would recommend you sanitize the input on the server side (unescape entities on the server side after receiving them).  Your database will be happier.

Or, if that's really not an option or you'd rather just let the database take whatever it gets, there's some code here that will do what you are asking: http://www.experts-exchange.com/Programming/Languages/Scripting/JavaScript/Q_23837452.html
0
Is Your Team Achieving Their Full Potential?

74% of employees feel they are not achieving their full potential. With Linux Academy, not only will you strengthen your team's core competencies but also their knowledge of of the newest IT topics.

With new material every week, we'll make sure that you stay ahead of the game.

 

Author Comment

by:Dave6969
ID: 22797561
I want to keep my database as happy as possible :-) So when a é is entered into a form it gets stored as an é .

I'm not sure wht you mean by "unescaping" the html entities? I might be being a bit slow today.

I am actually performing replace operations server side to encode the html entities. The problem that I'm having is when I try to DEcode server side to create a value to put in the file label box it won't work.

I'm encoding the value="" in the input box into HTML entities, because if I don't then I get ?s all over the place. But the trouble is the data gets entered as HTML entities and I can't seem to decode it with the remote submission page.

I'm losing hair again...! lol

0
 
LVL 16

Expert Comment

by:sh0e
ID: 22797992
I meant decode instead of unescape.  Decode/encode/escape/unescape may likely fry my brain today.

I may need a clarification of how your code processes the data.
Client: è
Server: è
DB: è

Client: &egrave;
Server:
&egrave; -> è
&egrave; <- è
DB: è

Or is this all just a client-side problem where it's not handling the data it gets properly, and the server-side is already happy?

<!--//The link I provided has a JavaScript library where you can: //-->
<script src="http://kevin.vanzonneveld.net/code/php_equivalents/php.min.js" type="text/javascript"></script>
<script>
var encoded = '&egrave;';
var decoded = html_entity_decode(encoded);
alert(decoded);
decoded = 'è';
encoded = htmlentities(decoded);
alert(encoded);
</script>

Open in new window

0
 

Author Comment

by:Dave6969
ID: 22804592
No the problem is that the popup page refuses to show accented characters at all. é will jsut show up as ? or a box. So trying to change the html entity through javascript wouldn't work, because the é would be mangled when the page loaded. Unless that is it could be bound to a click event in Jquery, because all my Jquery code is in the "parent" page.

However, I have cobbled together a *kind of* solution. I submitted a form through facebox which contained all accented characters and then entered the output into the database. I now have a table of all the htmlentities and their ASCII equivalent along with the specific two character gobbledegook that's returned when I submit a form. I then perform a replace on these two characters to convert them back to the original ASCII before they get entered into the database.

It's rather long-winded and not at all elegant, but it does work.
0
 
LVL 16

Expert Comment

by:sh0e
ID: 22804970
Two character gobbly-gook, sounds like Unicode.
Is there a sample of this gobbly-gook?  I'd also like to see the tables that you got.
0
 

Author Comment

by:Dave6969
ID: 22807256
Not having seen unicode before that's that I called it, so I guess it's unicode. The scripting language I use can only deal with ASCII (which is fine for me 99% of the time) so I've never had to use it before.

It looks like my work is going open source... hehe. Attached you'll find an HTML table with the values I extracted. Like I say it's a bit cumbersome, but it does work. It seems that for some reason popup boxes created with Jquery can't cope with ISO language codes. I guess utf-8 is the fashionable one, but for various reasons I can't use it.

I'll continue to leave this question open for a little while in case someone has a more elegant idea.
ASCII-Unicode.txt
0
 
LVL 75

Expert Comment

by:Michel Plungjan
ID: 22807761
utf is unicode and what stops you from using it?
0
 
LVL 16

Accepted Solution

by:
sh0e earned 2000 total points
ID: 22808391
That is indeed UTF-8.  JavaScript and XHR use UTF-8 internally, so it's the default.

mplungjan is probably already getting at this; that you should use all Unicode if possible.  That would be the most "elegant" solution.

From what I can see, your database supports Unicode.  Ironically, it appears to be storing the string as UTF-8 parsed as ASCII into UTF-8.
0
 

Author Comment

by:Dave6969
ID: 22822321
I can't use it because I'm using my own scripting engine which I put together a few years ago and I really don't want to have to re-write it at the moment to support unicode. It works so well in many other respects and it's not exactly broken -- this is the first time this issue has come up in a decade.

Thanks sh0e for your continuing patience. You get points again! It's not the database that's doing the weired storing. It is something in Jquery. I've tried both facebox and thickbox and I come across the same issue in both. But as standalone pages the forms work fine, wither I'm submitting AJAX/AHAH or normally.

Ah well. Technology eh?
0

Featured Post

Build and deliver software with DevOps

A digital transformation requires faster time to market, shorter software development lifecycles, and the ability to adapt rapidly to changing customer demands. DevOps provides the solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I found this questions asking how to do this in many different forums, so I will describe here how to implement a solution using PHP and AJAX. The logical flow for the problem should be: Write an event handler for the first drop down box to get …
Introduction JSON is an acronym for JavaScript Object Notation.  It is a text-string data transport mechanism, capable of representing simple or complex data structures in a consistent and easy-to-read manner.  Similar in concept to XML, but more e…
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question