Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Need to delete duplicate records

Posted on 2016-08-20
7
Medium Priority
?
58 Views
Last Modified: 2016-08-24
I was having problems with a table and I realized that there are some duplicate records that need to be deleted.
How do I do this if they are exact copies?

STOREKEY          |STORENUM|BNMB|STRT
NY1S05000010 |1001             |10       |Spring Street
NY1S05000010 |1001             |10       |Spring Street

I need to get rid of the 2nd one.  The dupes are not all over the file.  In other words, It only happens occasionally.
How it happened is beyond me.

Is there a quick script that I could write to get rid of these?
0
Comment
Question by:breeze351
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 9

Expert Comment

by:bas2754
ID: 41763851
I ran into this once and followed the guidance of this article to resolve:
-----
https://support.microsoft.com/en-us/kb/139444

I would advise testing on a test table first and ensure the results are what you expect.  Another expert may be able to provide exact commands for your situation, but this should get you headed in the right direction.
1
 
LVL 66

Expert Comment

by:Jim Horn
ID: 41763865
Let me know how this grabs ya:  SQL Server Delete Duplicate Rows Solutions.

Applied to your example and tested on my SQL 2012 box...
IF OBJECT_ID('tempdb..#tmp') IS NOT NULL
	DROP TABLE #tmp
GO

CREATE TABLE #tmp (STOREKEY varchar(20), STORENUM int, BNMB int, STRT varchar(20))
GO

INSERT INTO #tmp (STOREKEY, STORENUM, BNMB, STRT) 
VALUES 
   ('NY1S05000010', 1001, 10, 'Spring Street'), 
   ('NY1S05000010', 1001, 10, 'Spring Street'), -- a duplicate row
   ('TX1S05000010', 5963, 10, 'Main Street')  -- a row I just made up

-- Before removing the duplicates
SELECT * FROM #tmp

-- Delete the duplicates
;with cte as (
SELECT 
	STOREKEY, STORENUM, BNMB, STRT, 
	row_number() OVER (PARTITION BY STOREKEY, STORENUM, BNMB, STRT ORDER BY (SELECT NULL)) as row_number 
FROM #tmp) 
DELETE 
FROM cte
WHERE row_number > 1

-- After removing the duplicates
SELECT * FROM #tmp

 

Open in new window

3
 

Accepted Solution

by:
breeze351 earned 0 total points
ID: 41763880
Ok, I thought there might an easier way to do it.
I just write a record to a new table, if the key is there, I don't write.  When I'm done I no longer have dupes.
Thanks
Glenn
0
Survive A High-Traffic Event with Percona

Your application or website rely on your database to deliver information about products and services to your customers. You can’t afford to have your database lose performance, lose availability or become unresponsive – even for just a few minutes.

 

Author Comment

by:breeze351
ID: 41763901
I've requested that this question be closed as follows:

Accepted answer: 0 points for breeze351's comment #a41763880

for the following reason:

I had already thought of it.  Thought there might be an easier way
0
 
LVL 66

Expert Comment

by:Jim Horn
ID: 41763882
>Ok, I thought there might an easier way to do it.
Once the records are in the table, that's as easy as it gets.

>I just write a record to a new table, if the key is there, I don't write.
To prevent this from happening before the write you can create a unique index on the four columns, but there are other considerations in play such as memory, speed of insert/update, graceful way to handle violations, etc. that may not make this practical.

Another possibility is a trigger on the table to check inserts and updates, but if you're not comfortable with these then I'd recommend against them as it's added overhead.
0
 
LVL 51

Expert Comment

by:Mark Wills
ID: 41764353
>> How it happened is beyond me.

A couple of possibilities :
1) There is a unique identifier as a key
2) There is no unique index
3) unlikely, but possible that an ALTER TABLE with NOCHECK has happened at some stage to disable constraints

Looking at the data, it would appear that storekey should be enough to make that the unique / primary key.

So, once you clean up the duplicates, it would be worthwhile to then create a unique index :
create unique index PK_STOREKEY on <your table name> (STOREKEY)

Open in new window

or if STOREKEY is not nullable (if it is, the above code works), create a primary key
alter table <your table name> with nocheck add constraint PK_STOREKEY PRIMARY KEY (STOREKEY)

Open in new window


Hope that helps a bit more with ways to avoid it happening again.
1

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Ever needed a SQL 2008 Database replicated/mirrored/log shipped on another server but you can't take the downtime inflicted by initial snapshot or disconnect while T-logs are restored or mirror applied? You can use SQL Server Initialize from Backup…
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

718 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question