Solved

Identifying duplicates

Posted on 2016-09-25
3
47 Views
Last Modified: 2016-09-26
whats the easiest way to identify duplicates in the results of a sql query..
can you do it any other way besides organizing the results and then finding them visually
0
Comment
Question by:jamesmetcalf74
3 Comments
 
LVL 35

Assisted Solution

by:Terry Woods
Terry Woods earned 125 total points
ID: 41815098
Duplicate of one value (ie in one column), or multiple columns, or entire rows?

It can be done like this:
select col1, col2, count(*)
from my_table
group by col1, col2 having count(*)>1

Open in new window

1
 
LVL 18

Accepted Solution

by:
Pawan Kumar Khowal earned 250 total points
ID: 41815193
@Author - there are multiple options for this.

One of the option is to use left outer join.

CREATE TABLE DeleteDuplicateUsingOuterJoin
(
    ID SMALLINT
)
GO
 
INSERT INTO DeleteDuplicateUsingOuterJoin
VALUES
(1),
(2),
(2),
(3),
(3),
(3)
GO

;WITH CTE AS
(
    SELECT * , ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) rnk FROM DeleteDuplicateUsingOuterJoin
)
DELETE p
FROM CTE p
LEFT OUTER JOIN
(
    SELECT MIN(rnk) ID FROM CTE
    GROUP BY ID
)u ON u.ID = p.rnk
WHERE u.ID IS NULL

Open in new window




If you want to delete duplicate then use below-

@Author - there are multiple options for this. One of the option is

--CREATE TABLE
CREATE TABLE DeleteDup
(
ID INT
)
Go
 
--Insert Data
INSERT INTO DeleteDup VALUES(1),(2),(1)
 
--Verify Data
SELECT ID FROM DeleteDup

--
 
/************   SOLUTION 1         ****************/
 
 
 
WITH CTE AS
(
       SELECT *, ROW_NUMBER() OVER ( PARTITION BY ID ORDER BY ID ) RNK FROM DeleteDup    
)
DELETE FROM CTE WHERE RNK > 1
  
 
--

Open in new window

0
 
LVL 48

Assisted Solution

by:PortletPaul
PortletPaul earned 125 total points
ID: 41815196
Terry's answer above is a good common method.

Here is a EE article of the topic you may find helpful
https://www.experts-exchange.com/articles/3379/deleting-duplicate-records-in-a-database-without-a-primary-key.html

and note this comment under that article:
Deleting duplicates can be hard work, and I think you have covered off the more popular approaches pretty well.

I also like the importance of taking a backup first :)
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Introduction In my previous article (http://www.experts-exchange.com/Microsoft/Development/MS-SQL-Server/SSIS/A_9150-Loading-XML-Using-SSIS.html) I showed you how the XML Source component can be used to load XML files into a SQL Server database, us…
For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
Viewers will learn how the fundamental information of how to create a table.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now