Solved

ASP website showing weird characters

Posted on 2010-09-20
11
3,130 Views
Last Modified: 2012-05-10
Dear Experts,
we have a ASP portal running under IIS6. The databases behind runs on SQL Server. We've had the problem both on SQL Server 2008 and SQL Server 2000.
All of a sudden the portal started returning weird characters in place of French accented letters. For Example: "Jérôme" is now shown as "Jérôme".
The problem can be easily fixed by adding this line at the top of the asp page:
<meta HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">

Open in new window


But we don't want to modify all pages in the website.
If I execute the query directly on SQL Server console the name is shown correctly.
We haven't changed the collation of the database.

Why is this happening all of a sudden?
How can I fix it?

Server specs:
OS: Windows Server 2003 SP2
Web server: IIS 6
Database Server: SQL Server 2000

The server is up to date.
IIS and SQL Server reside on the same server.

Thanks,
Roberto.
0
Comment
Question by:martineit
  • 6
  • 5
11 Comments
 
LVL 21

Expert Comment

by:Dale Burrell
ID: 33714701
If you don't have the line <meta HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"> then the browser has to guess what character set you are using, and it appears to be getting it wrong from time to time.

You have two options that I know of, put that line into every asp file or I think you can add that as an HTTP header in IIS which is basically doing the same thing.
0
 
LVL 21

Expert Comment

by:Dale Burrell
ID: 33714709
0
 
LVL 1

Author Comment

by:martineit
ID: 33715789
Hi,
thanks for your prompt reply. What you said makes a lot of sense but I tried adding a "Custom HTTP Header" at different levels of the website but the meta tag is not added to the page when I make the request.
Moreover in one of the links you sent me it says:
'go to "Properties" => "HTTP Headers" => "File Types..." => "New Type...".' which doesn't exist in IIS6.
I changed that value in "Properties" => "HTTP Headers" => "Custom HTTP Header" => "Add"
Custom Header Name: http-equiv="content-type"
Custom Header Value: content="text/html;charset=utf-8"

Even so, it doesn't show in the Source of the page (among the Meta tags) nor the problem is fixed.
Is the "Custom HTTP Header" disabled by default?
If you try it yourself, do you see this value then reported in the source code of the page?

Thanks,
Roberto.
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 21

Expert Comment

by:Dale Burrell
ID: 33722538
HTTP headers are not the same as meta tags, and do not show in the source, but they should still have the desired effect. You can inspect the headers with something like Firebug for FIrefox or Fiddler.

Unfortunately I don't know anything more about adding the HTTP headers - all I know is that it can be done. I've always added the meta tag as thats the recommended way to do it.

You don't already have a meta tag specifying a different charset do you?

Hmmm... can't think of anything else of the top of my head
0
 
LVL 1

Author Comment

by:martineit
ID: 33723259
Yep, I tried with Fiddler and this is the header I get:
====================
HTTP/1.1 200 OK
Cache-Control: private
Connection: close
Date: Tue, 21 Sep 2010 08:17:55 GMT
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET1
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked
====================
I tried changing other parameters (X-Powered-By, I added 1 at the end) and it works, unfortunately although I set a Custom HTTP Header with: "Content-Type: text/html;charset=UTF-8" it still doesn't appear in the Header.

Any more ideas?
I checked on the internet and I can clearly see somebody managed to modify that part of the header but nobody says how...

Thanks,
Roberto.
0
 
LVL 1

Author Comment

by:martineit
ID: 33723590
One more comment on the subject:
Changing the HTTP Header works for any other file than ASP (and I guess ASP.NET too). Gif, css, html files all return a header like:
==========================
HTTP/1.1 304 Not Modified
Content-Type: text/html;charset=UTF-8
Last-Modified: Thu, 21 Sep 2006 15:51:59 GMT
Accept-Ranges: bytes
ETag: "84f04cdf95ddc61:1381"
Server: Microsoft-IIS/6.0
MicrosoftOfficeWebServer: 5.0_Pub
X-Powered-By: ASP.NET
Date: Tue, 21 Sep 2010 09:33:42 GMT
==========================
ASP default Content-Type cannot be changed from its default "text/html" (or so it seems).
Any clue or other solutions?

Thanks,
Roberto.
0
 
LVL 21

Expert Comment

by:Dale Burrell
ID: 33723841
I suspect that the the content-type is being set in the ASP code, which in turn produces an HTTP header, if your ASP pages include something like footer.asp then that would be an excellent place to set the header in ASP without having to modify every file.
0
 
LVL 1

Author Comment

by:martineit
ID: 33725195
Well, for the moment I wrote a Response.charset="utf-8" in the global.asa which seems to do the trick although I think it's quite dirty.
I still haven't figured out why making an explicit Response.charset="iso-8859-1" would still mess up French special characters while "UTF-8" doesn't.

Let's see if anybody knows something about this, otherwise I'll close the case and add some dirt to the global.asa.

Thanks,
Roberto.
0
 
LVL 21

Expert Comment

by:Dale Burrell
ID: 33731818
I imagine because its stored in your database using an nvarchar datatype which is a unicode format i.e. 2 bytes per character. So when you then tell the browser to display that using a single byte charset it shows all the interesting characters as two characters instead of one. It shows the regular chars as normal because they use the same single byte code, and the upper byte has nothing in it.

Its a pretty complex mechanism, there is how the chars are stored in the database, there is how the chars are handled by the code in ASP, then there is how the chars are interpreted by the browser. If any of those 3 stages doesn't match you get strange results as you are experiencing. The only reason I have any kind of understanding here is because we deliberately used the wrong charset on a system I worked on and tricked it to display it differently... not that you want to know that :)

The following might help if you are interested...

http://www.example-code.com/asp/asp-codepages.asp
http://en.wikipedia.org/wiki/Latin_characters_in_Unicode
http://msdn.microsoft.com/en-us/library/ms186939.aspx
0
 
LVL 1

Author Comment

by:martineit
ID: 33767864
Hey Dale,
that's a very interesting feedback. I'm still investigating this problem. I found that using a predefined include we use to connect to the database I get that problem while if I make a directo connection to the DB the characters are shown correctly.
More updates will follow.
By the way, I probably got it wrong but I would like to point out for other people who might read this post that changing the "Custom HTTP Header" at website level will mess up your website. All extensions will be presented with the same "content value" which means, for example, that zip files will be interpreted as html (instead of opening a download box). This problem only occurs with browsers other than IE.
0
 
LVL 1

Accepted Solution

by:
martineit earned 0 total points
ID: 33769924
Hello,
I finally found out what was the problem:
I created a simple file with a simple query to a database and it displayed the records correctly.
Then I started adding every include and at some stage I started getting the weird characters again.
I removed all includes and left the one causing the problem and I still had the problem.
I removed everything from within the file and I still had the problem.
I opened the file with a HEX editor (I used HHD Hex Editor Neo) and I could see the file was prefixed by a "ef bb bf" whose meaning you can find here: http://en.wikipedia.org/wiki/Byte_order_mark.
Once I removed it the special characters where displayed correctly again.
To remove that Hex string I had to open the file with notepad and copy the content of the file (not the file itself) in another notepad and then substitute the file.
If you want to search all files containing that Hex string you can use a Hex File Search (I used Effective File Search).

I hope this saves some time to someone else.

Thanks Dale for your inputs,
Roberto.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
IIS Cache Control settings 2 24
age from date of birth 4 51
Obtaining data on ASP 8 28
Designing forms 3 17
Lync server 2013 or Skype for business Backup Service Error ID 4049 – After File Share Migration
When it comes to showing a 404 error page to your visitors, you do not want that generic page to show, and you especially do not want your hosting provider’s ad error page to show either. In this article, I will show you how to enable the custom 40…
Finds all prime numbers in a range requested and places them in a public primes() array. I've demostrated a template size of 30 (2 * 3 * 5) but larger templates can be built such 210  (2 * 3 * 5 * 7) or 2310  (2 * 3 * 5 * 7 * 11). The larger templa…
In an interesting question (https://www.experts-exchange.com/questions/29008360/) here at Experts Exchange, a member asked how to split a single image into multiple images. The primary usage for this is to place many photographs on a flatbed scanner…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question