Go Premium for a chance to win a PS4. Enter to Win

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 4610
  • Last Modified:

SQL query to detect and delete duplicate records

I need a SQL query to detect and delete duplicate records, that is records where firstname and lastname are identical (if the case is different, would still be duplicate).  Detection and deletion could be in separate steps.
Thanks
0
MichaelMullin
Asked:
MichaelMullin
1 Solution
 
LowfatspreadCommented:
you need the Primary key of the row as well

select 'matched to ', t.pk, d.*
from  Table as D
inner Join Table as T
on D.Pk < T.PK
and D.Firstname=T.FirstName
and D.Lastname=T.lastName

you need criteria to decide which one to delete  

Delete from Table
Where Exists (Select T.pk from table as T
                      Where T.pk > Table.pk
                         and T.firstname=Table.firstname
                        and t.lastname = table.lastname)


I hope this isn't HOMEWORK ?    
0
 
BillAn1Commented:
If no primary key, try something like this :

SELECT DISTINCT firstname, lastname
INTO #temp_table
FROM source_table

DELETE FROM source_table

INSERT INTO source_table
SELECT * FROM #temp_table

0
 
LowfatspreadCommented:
do you have a case sensistivity problem ?

if so convert both names to upper case and then do the test....

0
 
arbertCommented:
Agree with lowfat--if you have something you can use for a key, you're better off to use that method.  If not, you need to use a method like BillAn1 suggested (I would just truncate the table instead of deleting the old rows--also, if you have a lot of data, this can be very slow) or a cursor.
0
 
Scott PletcherSenior DBACommented:
Here is a sample using a cursor but that does not require a separate table or a dump/reload:



DECLARE dupsCsr CURSOR READ_ONLY FOR
SELECT [firstName], [lastName], COUNT(*) AS numDups
FROM yourTable
GROUP BY [firstName], [lastName]
HAVING COUNT(*) > 1
DECLARE @firstName VARCHAR(30) --Change to match datatype on your table
DECLARE @lastName VARCHAR(30) --Change to match datatype on your table
DECLARE @numDups INT

OPEN dupsCsr
FETCH NEXT FROM dupsCsr INTO @firstName, @lastName, @numDups
WHILE @@FETCH_STATUS = 0
BEGIN
      SET @numDups = @numDups - 1 --delete all but 1 of the duplicates
      SET ROWCOUNT @numDups
      DELETE FROM yourTable
      WHERE [firstName] = @firstName AND [lastName] = @lastName
      FETCH NEXT FROM dupsCsr INTO @firstName, @lastName, @numDups
END --WHILE
CLOSE dupsCsr
DEALLOCATE dupsCsr

SET ROWCOUNT 0 --restore default
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now