Replacing accent characters when reading writing csv files

Posted on 2008-10-10
Last Modified: 2013-11-07
I am reading "certain" field values from several different csv files and combining these files into 1 csv file.  I have to format the values slightly before I add them to my master csv file because it will eventually be imported into SQL at a later date, mainly I just need to truncate the values so that the SQL import process won't complain.

The problem I am running into is when a value has accent characters.  For instance I have a value of
LA CITÉ.  So one of the things I do is replace É with E and if I MessageBox this, it works just as expeced.  However when I look at the master file after all the files have been processed, the value that actually got saved is LA CITÃ0 rather than LA CITE.

Any idea why this is happening and what I can do about it?  I am on .NET 1.1
Question by:GaryRasmussen
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2

Author Comment

ID: 22688402
I was wondering how those characters would show up in the question.  Where it says LA CITA0 in the last line, the A in CITA actually has a tilde over the top of it and the 0 in CITA is actually a fraction 0/00


Expert Comment

ID: 22688510
I think that will depend on the Character Encoding tag that you put in the html:

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

I don't know exactly how you change it in C#'s .ascx or aspx files.

Author Comment

ID: 22688542
The HTML?  This is a csv to csv import/export.  It is never rendered to HTML
Containers and Docker for Everyone

Containers are an incredibly powerful technology that can provide you and/or your engineering team with huge productivity gains. Using containers, you can deploy, back up, replicate, and move apps and their dependencies quickly and easily.


Expert Comment

ID: 22688557
Yeah... sorry about that... my bad.
LVL 26

Expert Comment

by:Anurag Thakur
ID: 22689012
is the following link going to be of any help to you

Accepted Solution

GaryRasmussen earned 0 total points
ID: 22689105

I read that one last night.  Unfortunately NormalizedString is only avaialble in .NET 2.0 and my application is .NET 1.1

I think the main issue here is that even when I look at the data in say Notepad and it looks like an E with an accent over the top of it, that is not neccessarily what the actual characters are because when the datatable reads the same value, it reads it as having other characters so there is no way to determine ahead of time what the actual string value is.

What is worse is that using substring to truncate the string doesn't work either because it treats the 4 chars that make up the single char as 1 character so the string does not get truncated to the correct length.

Featured Post

PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This document covers how to connect to SQL Server and browse its contents.  It is meant for those new to Visual Studio and/or working with Microsoft SQL Server.  It is not a guide to building SQL Server database connections in your code.  This is mo…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
This video Micro Tutorial shows how to password-protect PDF files with free software. Many software products can do this, such as Adobe Acrobat (but not Adobe Reader), Nuance PaperPort, and Nuance Power PDF, but they are not free products. This vide…
There are cases when e.g. an IT administrator wants to have full access and view into selected mailboxes on Exchange server, directly from his own email account in Outlook or Outlook Web Access. This proves useful when for example administrator want…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question