There are two ways that Unicode (specifically, Chinese) characters are getting entered into my Microsoft SQL database:
1. A user located in Taiwan is typing directly into a web form on a Windows computer that has Chinese as its default language, then my ASP script writes the result to the database.
2. A user located in the U.S. on a Macintosh has an Excel spreadsheet of language translations, which gets saved to a CSV file, uploaded to the server, then an ASP script processes that file and writes to the database. The actual characters in the Excel file come from a variety of sources: via e-mail from Taiwan, a Word or Excel file from Taiwan, or sometimes from Google Translate (via the web).
When I read from the database and display the text on a web page, here is what happens:
Using ASP, I insert this code at the top of my page: response.CharSet="UTF-8"
In the <head> area of the page, I also insert this: <meta charset="utf-8">
With only that, here is my test page:
The text from the U.S. computer is just question marks, the text from the Taiwan computer looks fine.
Next, I try adding another line of ASP code at the top of my page: response.CodePage=65001
Here is the result:
Now, the text from the U.S. computer looks fine, but the text from the the Taiwan computer is garbage.
From what I understand, response.CodePage=65001 is the same thing as UTF-8.
When I try selecting the text via a SQL Management Studio query window directly, I see the U.S. computer text looks fine, but the Taiwan computer text is garbage.
At this point, I am lost. I need to get a web page to display both of these types of text correctly, on the same page.
My assumption is that there is some difference between the character set on the U.S. computer vs. the Taiwan computer… that even though they are both Chinese, they are different versions of Chinese? If the data were exactly the same type, they should display the same on a web page.
Also, why adding response.CodePage=65001 flip-flops the success of those two types of text is a mystery to me.
Note that when the U.S. text goes bad, its all question marks. When the Taiwan text goes bad, its garbage characters.
This page shows eight different Chinese languages in the default SQL setup:http://msdn.microsoft.com/en-us/library/ms143508.aspx
… I don't know if that is related to my problem or not.
Would really appreciate any insight to this. Thank you!