In SQL replace unicode characters with latin characters

I import a UTF16 customer file from SAP BW, on a daily basis, into a staging table (SQL 2008R2).

We have a requirement to extract data without unicode characters.

I would like to replace unicode characters with the Latin equivalent.

For example:
à  ->  a
ś   ->  s
ï   ->   i
æ   ->  ae

Some data may be malformed and I'd also like to clean it up:

Sa³ley Ma¶ller  ->  Soley Moller
Trois-RiviÃ?¨res  -> Trois-Rivieres

Here is current collation:
SELECT SERVERPROPERTY('Collation')  --SQL_Latin1_General_CP1_CI_AS

Thank you for looking at this.
PetEdgeAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Aneesh RetnakaranDatabase AdministratorCommented:
I think you need to replace them manually

Update table  set Yourcolumn = replace( replace (YourColumn, 'à', 'a') , 'ś', 's')
0
Eugene ZCommented:
you may like to try this function

-- Removes special characters from a string value.
-- All characters except 0-9, a-z and A-Z are removed and
-- the remaining characters are returned.
-- Author: Christian d'Heureuse, www.source-code.biz
create function dbo.RemoveSpecialChars (@s varchar(256)) returns varchar(256)
   with schemabinding
begin
   if @s is null
      return null
   declare @s2 varchar(256)
   set @s2 = ''
   declare @l int
   set @l = len(@s)
   declare @p int
   set @p = 1
   while @p <= @l begin
      declare @c int
      set @c = ascii(substring(@s, @p, 1))
      if @c between 48 and 57 or @c between 65 and 90 or @c between 97 and 122
         set @s2 = @s2 + char(@c)
      set @p = @p + 1
      end
   if len(@s2) = 0
      return null
   return @s2
   end

Open in new window


-- how to use

CREATE TABLE [dbo].[t1](
	
	[NAME] [nvarchar](max) NULL
) ON [PRIMARY]

GO

insert into t1 (NAME)
Select 'Trois-Rivi�¨res'
select  dbo.RemoveSpecialChars (NAME) from t1 GO

Open in new window


--result TroisRivires
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
PetEdgeAuthor Commented:
Thank you very much!  I customized it a little and created a look up table and it's working very well.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.