String Manipulation -- REPLACE

In the last SQL post, I showed you SUBSTRING, and how you could use it to locate the area code in an un-formatted phone number.  The problem was the field we were working from was full of malformed numbers.  Today, we’ll start on cleaning up the numbers.

But first, I need to include my standard disclaimer:

I would like to point out that you have to be careful when and how often you use the techniques below.  A good rule to keep in mind is text manipulation is slow and painful to a server.  If you can leave the text manipulation to your middle-ware or front end, that would be better.  But we all have been stuck in a situation where we needed to alter a string before those two points, and so I bring you the following lesson.

And now back to our regularly scheduled post.

The Problem

phone
                      --------------------
                      605-555-2862
                      (561)555-2700
                      904-555-5680
                      N/A
                      580-555-5371
                      2815558368
                      (254)555-8430
                      336-555-2797
                      3365557233
                      592-555-3181/4951
                      96615551222 Ext. 249
                      +44 7930 555271
                      

Select all Open in new window

Looking back at our column, we can see there are a lot of different ways the phone number can be formed.  Before we begin trying to pull out the area codes, we need to clean the data.  In this situation, I usually go for sanitizing the column of anything that is not part of a phone number.  Given the phone number itself is just numbers; we want to remove anything that is not a number from this field.

There are two methods to accomplish this, the first is using the REPLACE function on any character (or string of characters) that is not a number.  Using only the REPLACE function requires you to know all the variations of data you wish to remove.  Since you can’t ever know everything you don’t know (I keep on trying though!), in the next post I’ll show you a little regular expression technique. The Solution

Let’s first look at the syntax for the REPLACE function.

REPLACE (searchString , findString , replaceWithString)
                      

Select all Open in new window

Using this function you can begin sanitizing the phone column.  Since we are going to be altering the source, I would advocate making a backup of the source to either a temp table, or make a full backup of the table before beginning.  Also,  when altering data, I am a big advocate of the following steps:

1

BEGIN TRANSACTION

2

SELECT the update you’re about to make

3

Make the UPDATE

4

SELECT the updated data and verify the results

5

If the results are correct:  COMMIT TRANSACTION

6

If the results are incorrect: ROLLBACK TRANSACTION

--BEGIN TRANSACTION --(in query analyzer/ssms you can highlight just "BEGIN TRANSACTION" to start the transaction.
                      --use replace to remove spaces
                      SELECT
                          Phone = REPLACE(phone, ' ', '') AS [Remove Space]
                      FROM sourceTable 
                      WHERE
                      	CHARINDEX(' ', phone) > 0
                      
                      /*
                      --the results
                      phone	                         Remove Space
                      ----------------------              ---------------------
                      +44 7930 555271	          +447930555271
                      96615551222 Ext. 249	  96615551222Ext.249
                      */
                      
                      UPDATE sourceTable SET
                         Phone = REPLACE(phone, ' ', '')
                      WHERE
                      	CHARINDEX(' ', phone) > 0
                      
                      --ROLLBACK TRANSACTION
                      --COMMIT TRANSACTION
                      

Select all Open in new window

Remember, you will have to select just the “BEGIN TRANSACTION “, “COMMIT TRANSACTION”, or “ROLLBACK TRANSACTION” as needed.  I leave them commented back, so it doesn’t auto-rollback after each run. That should remove spaces from the phone number; you can repeat those steps with variations to remove other bad data from phone.  Try the following to remove some more bad data from the phone field.

I'd also like to point out the CHARINDEX function.  In this example, the function finds the first space in the phone number.  The reason I've included this is I only want to run this update on rows that actually have a space in the data.  Those that do not have a space will be skipped over.  I'll cover the CHARINDEX in a later Article, since there are many ways you could use this function to your advantage.

SELECT 
                         phone
                         , REPLACE(phone, ' ', '') AS [Remove Space]
                         , REPLACE(phone, '+', '') AS [Remove +]
                         , REPLACE(phone, 'n/a', '') AS [Remove n/a]
                         , REPLACE(phone, '-', '')
                         , REPLACE(phone, 'ext.', '')
                      FROM sourceTable 
                      

Select all Open in new window

The following is what each of the above examples would do to the records.

phone                 Remove Space        Remove +              Remove na              Remove -               remove ext   
                      --------------------  ------------------  --------------------  ---------------------  ---------------------  ------------------
                      (254)555-8430         (254)555-8430       (254)555-8430         (254)555-8430          (254)5558430           (254)555-8430
                      (561)555-2700         (561)555-2700       (561)555-2700         (561)555-2700          (561)5552700           (561)555-2700
                      +44 7930 555271       +447930555271       44 7930 555271        +44 7930 555271        +44 7930 555271        +44 7930 555271
                      2815558368            2815558368          2815558368            2815558368             2815558368             2815558368
                      336-555-2797          336-555-2797        336-555-2797          336-555-2797           3365552797             336-555-2797
                      3365557233            3365557233          3365557233            3365557233             3365557233             3365557233
                      580-555-5371          580-555-5371        580-555-5371          580-555-5371           5805555371             580-555-5371
                      592-555-3181/4951     592-555-3181/4951   592-555-3181/4951     592-555-3181/4951      5925553181/4951        592-555-3181/4951
                      605-555-2862          605-555-2862        605-555-2862          605-555-2862           6055552862             605-555-2862
                      904-555-5680          904-555-5680        904-555-5680          904-555-5680           9045555680             904-555-5680
                      96615551222 Ext. 249  96615551222Ext.249  96615551222 Ext. 249  96615551222 Ext. 249   96615551222 Ext. 249   96615551222  249
                      N/A                   N/A                 N/A                                          N/A                    N/A
                      

Select all Open in new window

To proceed on this problem you’ll need a few more tools in your SQL toolbelt.  Check out other articles I’ve published on string manipulation.  There you will discover how LTRIM, RTRIM, and more techniques for manipulating text strings can help solve this problem.

References
http://msdn.microsoft.com/en-us/library/ms186862.aspx REPLACE-example-sourceTable.sql