Solved

Reading German special characters using a streamreader with currentEncoding = system.text.utf8encoding

Posted on 2007-11-17
4
1,501 Views
Last Modified: 2010-04-21
I'm using a streamreader to read a text file.

The streamreader.currentencoding is system.text.utf8encoding

The first line of my test text file is the following string of characters:
Uppercase A with an umlaut (HTML code would be Ä)
Lowercase a with an umlaut (ä)
Uppercase O with an umlaut (Ö)
Lowercase o with an umlaut (ö)
Uppercase U with an umlaut (Ü)
Lowercase u with an umlaut (ü)
The German lowercase "SZ" symbol (ß)

This is interpreted as a series of unknown characters.

What can I do to make the stream reader interpret the text correctly.

Thanks very much in advance,

JaimeHy
0
Comment
Question by:jaimehy
  • 2
  • 2
4 Comments
 
LVL 29

Accepted Solution

by:
Göran Andersson earned 500 total points
ID: 20304208
Then the file is obviously not encoded  using UTF-8. You have to specify the encoding that was used to create the text file.

If it's a ANSI (Windows pre-unicode) text file, the Encoding.Default property will give you the current ANSI encoding in the system, which will likely match the encoding used to create the file.

If the file is unicode, but not UTF-8, it usually has a BOM (byte order mark) at the beginning of the file. The StreamReader has a constructor that takes a boolean argument named detectEncodingFromByteOrderMarks that you can use to let the StreamReader select encoding based on the BOM.

If that fails, you can read the file using a FileStream, and examine the actual byte values in the file to see if there is any BOM, and what character codes you get for the special characters.
0
 
LVL 29

Expert Comment

by:Göran Andersson
ID: 20304217
As an alternative to using a FileStream to examine the file, just add a .bin file extension to the file and open it in Visual Studio.
0
 

Author Closing Comment

by:jaimehy
ID: 31409722
It gave me the pointers I needed to resolve the problem I had.  More than that I couldn't ask for.
0
 

Author Comment

by:jaimehy
ID: 20304412
Thanks for the tips, GreenGhost,

Yes, I did forget to mention that the txt file is encoded in ANSI.  Sorry about that.

The question that I suppose I wanted to ask is how to force the streamreader to read the file as ANSI.  With your help,I've managed to answer it.

The StreamReader.CurrentEncoding property is read only, so no luck there, and I was stuck.   What I eventually did was declare a default encoding and overload the arguments when declaring my streamreader as follows:

        Dim DefaultEncoding As Encoding = Encoding.Default
        Dim sr As New StreamReader(FileToBeRead, DefaultEncoding)

My umlauts now appear perfectly in my output xml file!

Much obliged


JaimeHY
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to create and use a custom WaterMark textbox class.  The custom WaterMark textbox class allows you to set the WaterMark Background Color and WaterMark text at design time.   IMAGE OF WATERMARKS STEPS Create VB …
Since .Net 2.0, Visual Basic has made it easy to create a splash screen and set it via the "Splash Screen" drop down in the Project Properties.  A splash screen set in this manner is automatically created, displayed and closed by the framework itsel…
Established in 1997, Technology Architects has become one of the most reputable technology solutions companies in the country. TA have been providing businesses with cost effective state-of-the-art solutions and unparalleled service that is designed…
Although Jacob Bernoulli (1654-1705) has been credited as the creator of "Binomial Distribution Table", Gottfried Leibniz (1646-1716) did his dissertation on the subject in 1666; Leibniz you may recall is the co-inventor of "Calculus" and beat Isaac…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question