In the last SQL post, I showed you SUBSTRING, and how you could use it to locate the area code in an un-formatted phone number. The problem was the field we were working from was full of malformed numbers. Today, we’ll start on cleaning up the numbers.
But first, I need to include my standard disclaimer:
I would like to point out that you have to be careful when and how often you use the techniques below. A good rule to keep in mind is text manipulation is slow and painful to a server. If you can leave the text manipulation to your middle-ware or front end, that would be better. But we all have been stuck in a situation where we needed to alter a string before those two points, and so I bring you the following lesson.
Looking back at our column, we can see there are a lot of different ways the phone number can be formed. Before we begin trying to pull out the area codes, we need to clean the data. In this situation, I usually go for sanitizing the column of anything that is not part of a phone number. Given the phone number itself is just numbers; we want to remove anything that is not a number from this field.
There are two methods to accomplish this, the first is using the REPLACE function on any character (or string of characters) that is not a number. Using only the REPLACE function requires you to know all the variations of data you wish to remove. Since you can’t ever know everything you don’t know (I keep on trying though!), in the next post I’ll show you a little regular expression technique. The Solution
Let’s first look at the syntax for the REPLACE function.
Using this function you can begin sanitizing the phone column. Since we are going to be altering the source, I would advocate making a backup of the source to either a temp table, or make a full backup of the table before beginning. Also, when altering data, I am a big advocate of the following steps:
SELECT the update you’re about to make
Make the UPDATE
SELECT the updated data and verify the results
If the results are correct: COMMIT TRANSACTION
If the results are incorrect: ROLLBACK TRANSACTION
--BEGIN TRANSACTION --(in query analyzer/ssms you can highlight just "BEGIN TRANSACTION" to start the transaction.--use replace to remove spacesSELECT Phone = REPLACE(phone, ' ', '') AS [Remove Space]FROM sourceTable WHERE CHARINDEX(' ', phone) > 0/*--the resultsphone Remove Space---------------------- ---------------------+44 7930 555271 +44793055527196615551222 Ext. 249 96615551222Ext.249*/UPDATE sourceTable SET Phone = REPLACE(phone, ' ', '')WHERE CHARINDEX(' ', phone) > 0--ROLLBACK TRANSACTION--COMMIT TRANSACTION
Remember, you will have to select just the “BEGIN TRANSACTION “, “COMMIT TRANSACTION”, or “ROLLBACK TRANSACTION” as needed. I leave them commented back, so it doesn’t auto-rollback after each run. That should remove spaces from the phone number; you can repeat those steps with variations to remove other bad data from phone. Try the following to remove some more bad data from the phone field.
I'd also like to point out the CHARINDEX function. In this example, the function finds the first space in the phone number. The reason I've included this is I only want to run this update on rows that actually have a space in the data. Those that do not have a space will be skipped over. I'll cover the CHARINDEX in a later Article, since there are many ways you could use this function to your advantage.
To proceed on this problem you’ll need a few more tools in your SQL toolbelt. Check out other articles I’ve published on string manipulation. There you will discover how LTRIM, RTRIM, and more techniques for manipulating text strings can help solve this problem.