Solved

Identifying duplicates

Posted on 2016-09-25
3
55 Views
Last Modified: 2016-09-26
whats the easiest way to identify duplicates in the results of a sql query..
can you do it any other way besides organizing the results and then finding them visually
0
Comment
Question by:jamesmetcalf74
3 Comments
 
LVL 35

Assisted Solution

by:Terry Woods
Terry Woods earned 125 total points
ID: 41815098
Duplicate of one value (ie in one column), or multiple columns, or entire rows?

It can be done like this:
select col1, col2, count(*)
from my_table
group by col1, col2 having count(*)>1

Open in new window

1
 
LVL 24

Accepted Solution

by:
Pawan Kumar earned 250 total points
ID: 41815193
@Author - there are multiple options for this.

One of the option is to use left outer join.

CREATE TABLE DeleteDuplicateUsingOuterJoin
(
    ID SMALLINT
)
GO
 
INSERT INTO DeleteDuplicateUsingOuterJoin
VALUES
(1),
(2),
(2),
(3),
(3),
(3)
GO

;WITH CTE AS
(
    SELECT * , ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) rnk FROM DeleteDuplicateUsingOuterJoin
)
DELETE p
FROM CTE p
LEFT OUTER JOIN
(
    SELECT MIN(rnk) ID FROM CTE
    GROUP BY ID
)u ON u.ID = p.rnk
WHERE u.ID IS NULL

Open in new window




If you want to delete duplicate then use below-

@Author - there are multiple options for this. One of the option is

--CREATE TABLE
CREATE TABLE DeleteDup
(
ID INT
)
Go
 
--Insert Data
INSERT INTO DeleteDup VALUES(1),(2),(1)
 
--Verify Data
SELECT ID FROM DeleteDup

--
 
/************   SOLUTION 1         ****************/
 
 
 
WITH CTE AS
(
       SELECT *, ROW_NUMBER() OVER ( PARTITION BY ID ORDER BY ID ) RNK FROM DeleteDup    
)
DELETE FROM CTE WHERE RNK > 1
  
 
--

Open in new window

0
 
LVL 48

Assisted Solution

by:PortletPaul
PortletPaul earned 125 total points
ID: 41815196
Terry's answer above is a good common method.

Here is a EE article of the topic you may find helpful
https://www.experts-exchange.com/articles/3379/deleting-duplicate-records-in-a-database-without-a-primary-key.html

and note this comment under that article:
Deleting duplicates can be hard work, and I think you have covered off the more popular approaches pretty well.

I also like the importance of taking a backup first :)
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For both online and offline retail, the cross-channel business is the most recent pattern in the B2C trade space.
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
Familiarize people with the process of retrieving data from SQL Server using an Access pass-thru query. Microsoft Access is a very powerful client/server development tool. One of the ways that you can retrieve data from a SQL Server is by using a pa…
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now