Solved

Another facebox issue: international characters not displayed

Posted on 2008-10-23
11
1,118 Views
Last Modified: 2012-08-13
Hello again experts

Thanks to sh0e's patience I got my previous issue sorted and I'm doing even more with facebox and everyone loves it. However I'm now going international and have some new files that need to be categorised in the database. The categorisation will all be in English but the file name and its associated label must remain in French.

I'm seeing an issue where all accented (diacritic) characters are displayed as ? or a box (depending on whether it's firefox or IE) when a facebox is opened. Now, I'm using ISO-8859-1 as my character set, as when I specify UTF-8 I get boxes everywhere there's an accent, not just in the facebox page. I'm wondering if my elderly version of ultraedit is to blame.

I'll post the code of the facebox page below, as I'm afraid I'm on a corporate intranet and can't really put demo a page fully online (yet). This page displays fine in its own browser window. Jquery is being properly loaded in the background window. I've removed the serverside code as I know it's working fine. The variables are encapsulated with $.

Also, I say it's a facebox problem, but a few moments ago I tried doing the same thing quickly using thickbox -- with the same results. So that's got me wondering where on earth I'm going wrong?

Hopefully someone can help. 500 points, as ever :-D


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://

www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="fr">

<head>

<meta http-equiv="content-type" content="text/

html;charset=ISO-8859-1" />
 

<style>

#filearticle {padding:0; margin:0}

#filearticle form {margin:0; padding: 0}

h1.articletitle {font-size: 10pt; color: #FFFFFF; background: #0077bb;

padding: 2px; margin: 0 }

#articlebody {padding: 5px}

#articlebody p {margin: 5px 0 5px 0}

#articlebody ul {margin: 5px 40px 5px 30px; list-style-type: square}

dl{

        width: 100%;

        padding: 0;

        background:#FFFFFF;

        margin: 0;

        text-align: left;

        clear: both;

        background: none;
 

}
 

dl dt{

        float: left;

        width: 25%;

        padding: .2em 0% 0.5em 0%;

        margin: 0 ;

        font-weight: bold;

        clear: left;

        font-size: 8pt;
 

}
 

dl dd{

        width: 70%;

        padding: .2em 0% 0.5em 0;

        float: left;

        margin: 0 ;

        font-size: 8pt;

}
 

input {border:1px solid #808080; background: #EEEEEE; font-size: 8pt;

font-family: Tahoma}

.quickalert1 {margin: 0;background: #0077bb; color: #FFFFFF; text-

align: center; font-size: 10pt; font-weight: normal; padding: 10px;

width: 60%;position: absolute; top: 35px ; left: 20%;border-top:1px

solid #99ccff; border-left:1px solid #99ccff;border-right:1px solid

#003366;border-bottom:1px solid #003366;}

#filelabelformsubmitted {display:none;}

</style>

<script type="text/javascript" src="js/jquery.form.min.js"></script>
 

<script type="text/javascript">
 

    $('#facebox #filelabelform').ajaxForm({

        target:       '#facebox #filelabelformsubmitted',

        beforeSubmit:  function(){

                       var filecategory1v = $

('select[@name=filecategory1]').fieldValue();

                       var filecategory2v = $

('select[@name=filecategory2]').fieldValue();

                       var filelanguagev  = $

('select[@name=filelanguage]').fieldValue();

                       var filepositionv  = $

('select[@name=fileposition]').fieldValue();
 

                       if (!filecategory1v[0] || !filecategory2v[0]

|| !filelanguagev[0] || !filepositionv[0]) {

                       $("#filelabelformsubmitted").html('You must

select from <b>ALL</b> dropdowns.').fadeIn('slow').animate({opacity:

1.0}, 2000).fadeOut('slow');

                       return false;

                       }

                       return true;
 

                       },

        success:       function(){

                       $('#facebox

#filelabelformsubmitted').fadeIn('slow').animate({opacity: 1.0},

2000).fadeOut('slow');

                       $('#facebox').animate({opacity: 1.0},

10000).fadeOut('slow');

                       $('#facebox_overlay').remove();

                       $("#filecat_$f

$").removeClass("categorylink").addClass("categorylinkdone");

                       }

    });

    </script>
 

</head>

<body>
 

<div id="filelabelformsubmitted" class="quickalert1"></div>

<div id="filearticle">

<h1 class="facebox_title">Categorise File</h1>

<div id="articlebody">
 

<form id="filelabelform" method="post" action="filelabelscript.dpe">
 

<dl>

<!--NOTE : when filename is displayed below it has ? or square for

accented characters depending on browser.-->

<dt>File name:</dt><dd>$filename$</dd>

<dt>File entity:</dt><dd>$entity$</dd>

<input type="hidden" name="filename" value="$filename$">

<input type="hidden" name="entity" value="$entity$">

<dt>File label:</dt><dd><input type="text" name="filelabel" size="40"

value="$filelabel1$" id="filelabel"></dd>

<dt>File Category1:</dt><dd><select size="1" name="filecategory1"

id="filecategory1">
 

                               <option value="">Select...</option>
 

                               <option value="$id$" selected>$category

$</option>
 

                              </select></dd>
 

<dt>File Category2:</dt><dd><select size="1" name="filecategory2"

id="filecategory2">
 

                               <option value="">Select...</option>
 

                               <option value="$id$" selected>$category

$</option>
 

                              </select></dd>
 

<dt>File Language:</dt><dd><select size="1" name="filelanguage"

id="filelanguage">
 

                               <option value="">Select...</option>
 

                               <option value="$id$" selected>$language

$</option>
 

                              </select></dd>

<dt>File Positioning:</dt><dd><select size="1" name="fileposition"

id="fileposition">
 

                               <option value="">Select...</option>
 

                               <option value="$id$" selected>$position

$</option>
 

                              </select>
 

</dd>

<dt></dt><dd><input type="submit" value="submit" id="btnSubmit"></dd>

</dl>

</form>
 

</div>

</div>

</body>

</html>

Open in new window

0
Comment
Question by:Dave6969
  • 5
  • 4
  • 2
11 Comments
 
LVL 75

Expert Comment

by:Michel Plungjan
ID: 22795371
If the characters are saved as DOS iso-8859-1 then you need iso-8859-1
If they are saved as UTF8, you need UTF8
If you use entities (&egrave; &euml;) then it does not matter
0
 

Author Comment

by:Dave6969
ID: 22796634
Actually since no one was answering I got to grips with it a bit. I've done some replacing server side and I can get it to display fine.

HOWEVER, there is a form field in the page for the file label. It's going to be mostly english speakers categorising this as first. If you type accented characters into the field (as a french speaker would do) there's no problem submitting, but if you copy and paste (as an english speaker will do) from the displayed filename (with the HTML entities in it) then a load of gobbledegook gets entered into the database where the HTML entities used to be.

I tried writing some jquery code to replace the entities on-the-fly, but of course I hit the same problem with the characters not displaying in the page. Very frustrating!
0
 
LVL 16

Expert Comment

by:sh0e
ID: 22796825
It's already done, but usually the solution seems to be to force encoding on the server side.

It sounds like you are trying to sanitize entities on the client side.  I would recommend you sanitize the input on the server side (unescape entities on the server side after receiving them).  Your database will be happier.

Or, if that's really not an option or you'd rather just let the database take whatever it gets, there's some code here that will do what you are asking: http://www.experts-exchange.com/Programming/Languages/Scripting/JavaScript/Q_23837452.html
0
 

Author Comment

by:Dave6969
ID: 22797561
I want to keep my database as happy as possible :-) So when a é is entered into a form it gets stored as an é .

I'm not sure wht you mean by "unescaping" the html entities? I might be being a bit slow today.

I am actually performing replace operations server side to encode the html entities. The problem that I'm having is when I try to DEcode server side to create a value to put in the file label box it won't work.

I'm encoding the value="" in the input box into HTML entities, because if I don't then I get ?s all over the place. But the trouble is the data gets entered as HTML entities and I can't seem to decode it with the remote submission page.

I'm losing hair again...! lol

0
 
LVL 16

Expert Comment

by:sh0e
ID: 22797992
I meant decode instead of unescape.  Decode/encode/escape/unescape may likely fry my brain today.

I may need a clarification of how your code processes the data.
Client: è
Server: è
DB: è

Client: &egrave;
Server:
&egrave; -> è
&egrave; <- è
DB: è

Or is this all just a client-side problem where it's not handling the data it gets properly, and the server-side is already happy?

<!--//The link I provided has a JavaScript library where you can: //-->

<script src="http://kevin.vanzonneveld.net/code/php_equivalents/php.min.js" type="text/javascript"></script>

<script>

var encoded = '&egrave;';

var decoded = html_entity_decode(encoded);

alert(decoded);

decoded = 'è';

encoded = htmlentities(decoded);

alert(encoded);

</script>

Open in new window

0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:Dave6969
ID: 22804592
No the problem is that the popup page refuses to show accented characters at all. é will jsut show up as ? or a box. So trying to change the html entity through javascript wouldn't work, because the é would be mangled when the page loaded. Unless that is it could be bound to a click event in Jquery, because all my Jquery code is in the "parent" page.

However, I have cobbled together a *kind of* solution. I submitted a form through facebox which contained all accented characters and then entered the output into the database. I now have a table of all the htmlentities and their ASCII equivalent along with the specific two character gobbledegook that's returned when I submit a form. I then perform a replace on these two characters to convert them back to the original ASCII before they get entered into the database.

It's rather long-winded and not at all elegant, but it does work.
0
 
LVL 16

Expert Comment

by:sh0e
ID: 22804970
Two character gobbly-gook, sounds like Unicode.
Is there a sample of this gobbly-gook?  I'd also like to see the tables that you got.
0
 

Author Comment

by:Dave6969
ID: 22807256
Not having seen unicode before that's that I called it, so I guess it's unicode. The scripting language I use can only deal with ASCII (which is fine for me 99% of the time) so I've never had to use it before.

It looks like my work is going open source... hehe. Attached you'll find an HTML table with the values I extracted. Like I say it's a bit cumbersome, but it does work. It seems that for some reason popup boxes created with Jquery can't cope with ISO language codes. I guess utf-8 is the fashionable one, but for various reasons I can't use it.

I'll continue to leave this question open for a little while in case someone has a more elegant idea.
ASCII-Unicode.txt
0
 
LVL 75

Expert Comment

by:Michel Plungjan
ID: 22807761
utf is unicode and what stops you from using it?
0
 
LVL 16

Accepted Solution

by:
sh0e earned 500 total points
ID: 22808391
That is indeed UTF-8.  JavaScript and XHR use UTF-8 internally, so it's the default.

mplungjan is probably already getting at this; that you should use all Unicode if possible.  That would be the most "elegant" solution.

From what I can see, your database supports Unicode.  Ironically, it appears to be storing the string as UTF-8 parsed as ASCII into UTF-8.
0
 

Author Comment

by:Dave6969
ID: 22822321
I can't use it because I'm using my own scripting engine which I put together a few years ago and I really don't want to have to re-write it at the moment to support unicode. It works so well in many other respects and it's not exactly broken -- this is the first time this issue has come up in a decade.

Thanks sh0e for your continuing patience. You get points again! It's not the database that's doing the weired storing. It is something in Jquery. I've tried both facebox and thickbox and I come across the same issue in both. But as standalone pages the forms work fine, wither I'm submitting AJAX/AHAH or normally.

Ah well. Technology eh?
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

PROBLEM: The other day I was working on adding an ajax request to a webpage that already had a dialog box on the page.  The dialog box was using relative positioning to be positioned next to a form field I had on the page.  Everything was working…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
The viewer will learn the basics of jQuery, including how to invoke it on a web page. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery.: (CODE)
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now