Solved

SP to delete duplicates

Posted on 2016-09-27
15
60 Views
Last Modified: 2016-09-28
I have a table "Atts" which holds a few values, there maybe duplicate entries. I need a stored procedure that will remove any duplicate entries.

This is the table

SELECT AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts FROM dbo.Atts

Open in new window


any records which are identical after running the SP only one should remain, any records without duplicates should stay as is.
0
Comment
Question by:amucinobluedot
  • 8
  • 3
  • 2
  • +2
15 Comments
 
LVL 32

Expert Comment

by:Big Monty
ID: 41818811
assuming that AttsID is where the duplicates are, try:

WITH CTE AS(
   SELECTAttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts
       RN = ROW_NUMBER()OVER(PARTITION BY AttsId ORDER BY AttsId )
   FROM dbo.Atts
)
DELETE FROM CTE WHERE RN > 1

Open in new window


try running the code directly in your database (maybe use a test table before anything). if that works, then stick in a SP and you'll be good to go
0
 

Author Comment

by:amucinobluedot
ID: 41818822
Actually the attsid is unique. The duplicate entries would be userid and caseid

So:

AttsId - Unique key
FirmId , -- irrelevant
CaseId , check 1
UserId ,  check 2
AccessLevel ,-- irrelevant
Xfer2Forms ,-- irrelevant
Ordernumatts -- irrelevant

I need to make sure that there is no more than one entry where userid and caseid are the same in more than one row.

if userid = 100
and caseid = 45

and there is another rown with those same values then its a duplicate. If they are different then not duplicate. The rest of the values are irrelevant. most of the time they are all the same for all rows other than the first one which is the ID and its unique and shouldn't be compared. Hope this clarifies things.
0
 
LVL 34

Expert Comment

by:Brian Crowe
ID: 41818928
Use Monty's RowNumber solution but PARTITION BY userid and caseid instead of attsid
0
 

Author Comment

by:amucinobluedot
ID: 41818930
Not sure how to do that with two items I need to check and only when BOTH are the same is considered duplicate.
0
 
LVL 69

Accepted Solution

by:
ScottPletcher earned 250 total points
ID: 41818938
;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1
0
 
LVL 24

Expert Comment

by:Pawan Kumar
ID: 41818939
Use below code.. Enjoy !!

WITH CTE AS
(
   SELECT
	   AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts
       ROW_NUMBER()OVER(PARTITION BY UserId,CaseId ORDER BY (SELECT NULL)) rnk
   FROM dbo.Atts
)
DELETE FROM CTE WHERE rnk > 1

Open in new window

0
 

Author Comment

by:amucinobluedot
ID: 41818948
I'll try both as soon as I get home ...
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 

Author Comment

by:amucinobluedot
ID: 41818962
For some reason Pawan's script gives an error:

Msg 102, Level 15, State 1, Line 20
Incorrect syntax near '('.
Msg 102, Level 15, State 1, Line 20
Incorrect syntax near ')'.


the one from Scott seemed to work fine. Ill test some more.
0
 
LVL 24

Assisted Solution

by:Pawan Kumar
Pawan Kumar earned 250 total points
ID: 41818965
comma was missing. chk this out.

;WITH CTE AS
(
   SELECT
	   AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts,
       ROW_NUMBER()OVER(PARTITION BY UserId,CaseId ORDER BY (SELECT NULL)) rnk
   FROM dbo.Atts
)
DELETE FROM CTE WHERE rnk > 1

Open in new window

0
 

Author Comment

by:amucinobluedot
ID: 41818966
Yep, that works. ill assing points to both. Thanks.
0
 

Author Closing Comment

by:amucinobluedot
ID: 41818967
Thanks to both.
0
 

Author Comment

by:amucinobluedot
ID: 41818979
I tried to create the SP in my Db but I get an error. I tried this:

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]

;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1

Open in new window


Error:
Msg 195, Level 15, State 1, Procedure admin_removeduplicate_links, Line 5 [Batch Start Line 7]
'cte_dups' is not a recognized option.


I am using MS SQL 2008R2
0
 
LVL 24

Expert Comment

by:Pawan Kumar
ID: 41818981
Here it is

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]
AS

BEGIN

	;WITH cte_dups AS (
		SELECT
			AttsId,
			ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
		FROM dbo.Atts
	)
	DELETE FROM cte_dups
	WHERE row_num > 1

END

Open in new window

0
 

Author Comment

by:amucinobluedot
ID: 41818982
Thx!
0
 
LVL 69

Expert Comment

by:ScottPletcher
ID: 41820049
I think all you really have to have is the word "AS".  I don't add an extra "BEGIN" unless I really need it.

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]
AS
;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Although it can be difficult to imagine, someday your child will have a career of his or her own. He or she will likely start a family, buy a home and start having their own children. So, while being a kid is still extremely important, it’s also …
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

912 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now