Solved

using streamReader to read file with special characters such as copyright, registered

Posted on 2013-01-30
6
603 Views
Last Modified: 2013-02-04
I am reading a text file using the StreamReader with lines that have special characters such as copyright or registered
trademark symbol.
In the Model the property CompDescription is a string and I assign to it
from the file, the following sentence -->
StatLock® Stabilization Device
but for some reason when I debug and go into immediate window I see
StatLock¿ Stabilization Device and later when I save to the DB I see
'StatLock? Stabilization Device'

Below is the code.

 
var detailReader = reader.AsEnumerable();
 dataImportModel.WorkOrderDetail = (from line in detailReader
 select new WorkOrderDetailModel
 {
 Company = Int16.Parse(line.Substring(0, 3).Trim()),
 WorkOrderID = line.Substring(3, 7).Trim(),
 Component = string.Format("{0}", line.Substring(109, 15).Trim()),
 CompDescription = line.Substring(172, 1).Trim().Equals("N") ? "DELETE" :
 Convert.ToString(string.IsNullOrEmpty(line.Substring(173, 30).Trim()) ? string.Empty :
 line.Substring(173, 30).Trim()) + "|" + Convert.ToString(string.IsNullOrEmpty(line.Substring(281, 150).Trim()) ? string.Empty :
 line.Substring(281, 150).Trim()),
 QtyPerKit = decimal.Parse(line.Substring(124, 16).Trim()),
 UoM = line.Substring(203, 2).Trim(),
 Disclaimers = string.Empty,
 Deviation = line.Substring(209, 1).Trim(),
 Sequence = line.Substring(431, 1).Trim(),
 LevelOneBOMFlg = line.Substring(432, 3).Trim(),
 ExcludeFlg = line.Substring(465, 1).Trim()
 }).ToList();
 using (var reader = new StreamReader(fileName))
                {
                    try
                    {
                        
                       //while (reader.Peek() >= 0)
                       //{
                       //         Console.WriteLine(reader.ReadLine());
                       //         string strRecord
                                
                       //}
                        
                        var detailReader = reader.AsEnumerable();
                        dataImportModel.WorkOrderDetail = (from line in detailReader
                                                           select new WorkOrderDetailModel
                                                           {
                                                               Company = Int16.Parse(line.Substring(0, 3).Trim()),
                                                               WorkOrderID = line.Substring(3, 7).Trim(),
                                                               Component = string.Format("{0}", line.Substring(109, 15).Trim()),
                                                               CompDescription = line.Substring(172, 1).Trim().Equals("N") ? "DELETE" :
                                                                                                                     Convert.ToString(string.IsNullOrEmpty(line.Substring(173, 30).Trim()) ? string.Empty :
                                                                                                                     line.Substring(173, 30).Trim()) + "|" + Convert.ToString(string.IsNullOrEmpty(line.Substring(281, 150).Trim()) ? string.Empty :
                                                                                                                     line.Substring(281, 150).Trim()),
                                                               QtyPerKit = decimal.Parse(line.Substring(124, 16).Trim()),
                                                               UoM = line.Substring(203, 2).Trim(),
                                                               Disclaimers = string.Empty,
                                                               Deviation = line.Substring(209, 1).Trim(),
                                                               Sequence = line.Substring(431, 1).Trim(),
                                                               LevelOneBOMFlg = line.Substring(432, 3).Trim(),
                                                               ExcludeFlg = line.Substring(465, 1).Trim()
                                                           }).ToList();
                    }

Open in new window

0
Comment
Question by:mbarazi
6 Comments
 
LVL 79

Expert Comment

by:David Johnson, CD, MVP
ID: 38839005
using (var reader = new StreamReader(Filename, System.Text.Encoding.ANSI); // or UTF-8
0
 

Author Comment

by:mbarazi
ID: 38839694
Hi,

I Tried UTF-8 and still it shows StatLock? Stabilization Device and ANSI is not available as an option.
The following is what it should show in debugger  
StatLock® Stabilization Device
0
 
LVL 23

Accepted Solution

by:
Roopesh Reddy earned 500 total points
ID: 38839859
Hi,

Did you tried Encoding.Default ?

Hope it helps u...
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 
LVL 16

Expert Comment

by:hjgode
ID: 38840681
The encoding to use for streamreader depends on the encoding used in the text file. Do you know the encoding of the text file?

The encoding used in the VS output window is Unicode. An unicode encoded file should start with a BOM (byte order marker) but also UTF-8 etc can have BOM: http://en.wikipedia.org/wiki/Byte_order_mark

If the file is ANSI that will be now 1252 encoding: see http://msdn.microsoft.com/en-us/library/ms404377.aspx

Encoding and decoding extended chars into databases depends on the correct encoding setting of the database and the encoding used during add/read data. If the database is prepared for 1252 encoding and you store Unicode data but then read it back, you will get the wrong mapping.
0
 
LVL 23

Expert Comment

by:Roopesh Reddy
ID: 38849922
Hi,

Is the issue solved?
0
 

Author Closing Comment

by:mbarazi
ID: 38852636
That did Work, thanks for the help.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

IP addresses can be stored in a database in any of several ways.  These ways may vary based on the volume of the data.  I was dealing with quite a large amount of data for user authentication purpose, and needed a way to minimize the storage.   …
A long time ago (May 2011), I have written an article showing you how to create a DLL using Visual Studio 2005 to be hosted in SQL Server 2005. That was valid at that time and it is still valid if you are still using these versions. You can still re…
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question