Solved

deduplicate SQL 2005 table on one field

Posted on 2014-01-06
2
389 Views
Last Modified: 2014-01-06
hello all! I have a table with 3 fields, let's just call them field1, email, and field3. So I have duplicate email addresses throughout the table. I want to deduplicate this table based on email. I don't care what records are removed as long as the table is trimmed down to non-duplicate email addresses. Is there a way I can do this? Thanks!
0
Comment
Question by:mrosier
2 Comments
 
LVL 59

Accepted Solution

by:
Kevin Cross earned 500 total points
ID: 39759617
Yes, there are several ways to accomplish this.  One way, leveraging the ROW_NUMBER() and OVER() clause in SQL 2005, is to rank your records 1 to n within a derived query or common table expression, then DELETE records with rank > 1 in outer query.

Using this method, you can use the ordering of the other pieces of information to determine the retention.  For example, if field1 is an integer that increments, you may want the lowest ID value (i.e., the first entry).  Therefore, you could ORDER BY field1.  If you want the last, you could ORDER BY field1 DESC.  If it truly does not matter, you can make the ORDER BY random.

For example:
;WITH cte(field1, email, field3, [rank]) AS (
    SELECT field1, email, field3
         , ROW_NUMBER() OVER(PARTITION BY email ORDER BY field1)
    FROM your_table_name
)
DELETE
FROM cte
WHERE [rank] > 1
;

Open in new window


I hope that helps!

P.S. I would run the inner query first, so you understand what the ranking does.  Second, you can change "DELETE" to "SELECT field1, email, field3, [rank]" so you can see what records will be removed before issuing the deletion.

Best regards,

Kevin
0
 

Author Comment

by:mrosier
ID: 39759667
excellent, it really is truly a non-issue as to what records stay as long as email addresses in the table are unique. And that is how it usually is with data deduplication for me. I appreciate the help, thanks!
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

I am showing a way to read/import the excel data in table using SQL server 2005... Suppose there is an Excel file "Book1" at location "C:\temp" with column "First Name" and "Last Name". Now to import this Excel data into the table, we will use…
In SQL Server, when rows are selected from a table, does it retrieve data in the order in which it is inserted?  Many believe this is the case. Let us try to examine for ourselves with an example. To get started, use the following script, wh…
This video discusses moving either the default database or any database to a new volume.
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now