using streamReader to read file with special characters such as copyright, registered

I am reading a text file using the StreamReader with lines that have special characters such as copyright or registered
trademark symbol.
In the Model the property CompDescription is a string and I assign to it
from the file, the following sentence -->
StatLock® Stabilization Device
but for some reason when I debug and go into immediate window I see
StatLock┬┐ Stabilization Device and later when I save to the DB I see
'StatLock? Stabilization Device'

Below is the code.

 
var detailReader = reader.AsEnumerable();
 dataImportModel.WorkOrderDetail = (from line in detailReader
 select new WorkOrderDetailModel
 {
 Company = Int16.Parse(line.Substring(0, 3).Trim()),
 WorkOrderID = line.Substring(3, 7).Trim(),
 Component = string.Format("{0}", line.Substring(109, 15).Trim()),
 CompDescription = line.Substring(172, 1).Trim().Equals("N") ? "DELETE" :
 Convert.ToString(string.IsNullOrEmpty(line.Substring(173, 30).Trim()) ? string.Empty :
 line.Substring(173, 30).Trim()) + "|" + Convert.ToString(string.IsNullOrEmpty(line.Substring(281, 150).Trim()) ? string.Empty :
 line.Substring(281, 150).Trim()),
 QtyPerKit = decimal.Parse(line.Substring(124, 16).Trim()),
 UoM = line.Substring(203, 2).Trim(),
 Disclaimers = string.Empty,
 Deviation = line.Substring(209, 1).Trim(),
 Sequence = line.Substring(431, 1).Trim(),
 LevelOneBOMFlg = line.Substring(432, 3).Trim(),
 ExcludeFlg = line.Substring(465, 1).Trim()
 }).ToList();
 using (var reader = new StreamReader(fileName))
                {
                    try
                    {
                        
                       //while (reader.Peek() >= 0)
                       //{
                       //         Console.WriteLine(reader.ReadLine());
                       //         string strRecord
                                
                       //}
                        
                        var detailReader = reader.AsEnumerable();
                        dataImportModel.WorkOrderDetail = (from line in detailReader
                                                           select new WorkOrderDetailModel
                                                           {
                                                               Company = Int16.Parse(line.Substring(0, 3).Trim()),
                                                               WorkOrderID = line.Substring(3, 7).Trim(),
                                                               Component = string.Format("{0}", line.Substring(109, 15).Trim()),
                                                               CompDescription = line.Substring(172, 1).Trim().Equals("N") ? "DELETE" :
                                                                                                                     Convert.ToString(string.IsNullOrEmpty(line.Substring(173, 30).Trim()) ? string.Empty :
                                                                                                                     line.Substring(173, 30).Trim()) + "|" + Convert.ToString(string.IsNullOrEmpty(line.Substring(281, 150).Trim()) ? string.Empty :
                                                                                                                     line.Substring(281, 150).Trim()),
                                                               QtyPerKit = decimal.Parse(line.Substring(124, 16).Trim()),
                                                               UoM = line.Substring(203, 2).Trim(),
                                                               Disclaimers = string.Empty,
                                                               Deviation = line.Substring(209, 1).Trim(),
                                                               Sequence = line.Substring(431, 1).Trim(),
                                                               LevelOneBOMFlg = line.Substring(432, 3).Trim(),
                                                               ExcludeFlg = line.Substring(465, 1).Trim()
                                                           }).ToList();
                    }

Open in new window

mbaraziAsked:
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

x
 
Roopesh ReddyConnect With a Mentor IT AnalystCommented:
Hi,

Did you tried Encoding.Default ?

Hope it helps u...
0
 
David Johnson, CD, MVPOwnerCommented:
using (var reader = new StreamReader(Filename, System.Text.Encoding.ANSI); // or UTF-8
0
 
mbaraziAuthor Commented:
Hi,

I Tried UTF-8 and still it shows StatLock? Stabilization Device and ANSI is not available as an option.
The following is what it should show in debugger  
StatLock® Stabilization Device
0
Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

 
hjgodeCommented:
The encoding to use for streamreader depends on the encoding used in the text file. Do you know the encoding of the text file?

The encoding used in the VS output window is Unicode. An unicode encoded file should start with a BOM (byte order marker) but also UTF-8 etc can have BOM: http://en.wikipedia.org/wiki/Byte_order_mark

If the file is ANSI that will be now 1252 encoding: see http://msdn.microsoft.com/en-us/library/ms404377.aspx

Encoding and decoding extended chars into databases depends on the correct encoding setting of the database and the encoding used during add/read data. If the database is prepared for 1252 encoding and you store Unicode data but then read it back, you will get the wrong mapping.
0
 
Roopesh ReddyIT AnalystCommented:
Hi,

Is the issue solved?
0
 
mbaraziAuthor Commented:
That did Work, thanks for the help.
0
All Courses

From novice to tech pro — start learning today.