Solved

SP to delete duplicates

Posted on 2016-09-27
15
54 Views
Last Modified: 2016-09-28
I have a table "Atts" which holds a few values, there maybe duplicate entries. I need a stored procedure that will remove any duplicate entries.

This is the table

SELECT AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts FROM dbo.Atts

Open in new window


any records which are identical after running the SP only one should remain, any records without duplicates should stay as is.
0
Comment
Question by:amucinobluedot
  • 8
  • 3
  • 2
  • +2
15 Comments
 
LVL 32

Expert Comment

by:Big Monty
Comment Utility
assuming that AttsID is where the duplicates are, try:

WITH CTE AS(
   SELECTAttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts
       RN = ROW_NUMBER()OVER(PARTITION BY AttsId ORDER BY AttsId )
   FROM dbo.Atts
)
DELETE FROM CTE WHERE RN > 1

Open in new window


try running the code directly in your database (maybe use a test table before anything). if that works, then stick in a SP and you'll be good to go
0
 

Author Comment

by:amucinobluedot
Comment Utility
Actually the attsid is unique. The duplicate entries would be userid and caseid

So:

AttsId - Unique key
FirmId , -- irrelevant
CaseId , check 1
UserId ,  check 2
AccessLevel ,-- irrelevant
Xfer2Forms ,-- irrelevant
Ordernumatts -- irrelevant

I need to make sure that there is no more than one entry where userid and caseid are the same in more than one row.

if userid = 100
and caseid = 45

and there is another rown with those same values then its a duplicate. If they are different then not duplicate. The rest of the values are irrelevant. most of the time they are all the same for all rows other than the first one which is the ID and its unique and shouldn't be compared. Hope this clarifies things.
0
 
LVL 34

Expert Comment

by:Brian Crowe
Comment Utility
Use Monty's RowNumber solution but PARTITION BY userid and caseid instead of attsid
0
 

Author Comment

by:amucinobluedot
Comment Utility
Not sure how to do that with two items I need to check and only when BOTH are the same is considered duplicate.
0
 
LVL 69

Accepted Solution

by:
ScottPletcher earned 250 total points
Comment Utility
;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1
0
 
LVL 17

Expert Comment

by:Pawan Kumar Khowal
Comment Utility
Use below code.. Enjoy !!

WITH CTE AS
(
   SELECT
	   AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts
       ROW_NUMBER()OVER(PARTITION BY UserId,CaseId ORDER BY (SELECT NULL)) rnk
   FROM dbo.Atts
)
DELETE FROM CTE WHERE rnk > 1

Open in new window

0
 

Author Comment

by:amucinobluedot
Comment Utility
I'll try both as soon as I get home ...
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 

Author Comment

by:amucinobluedot
Comment Utility
For some reason Pawan's script gives an error:

Msg 102, Level 15, State 1, Line 20
Incorrect syntax near '('.
Msg 102, Level 15, State 1, Line 20
Incorrect syntax near ')'.


the one from Scott seemed to work fine. Ill test some more.
0
 
LVL 17

Assisted Solution

by:Pawan Kumar Khowal
Pawan Kumar Khowal earned 250 total points
Comment Utility
comma was missing. chk this out.

;WITH CTE AS
(
   SELECT
	   AttsId ,
       FirmId ,
       CaseId ,
       UserId ,
       AccessLevel ,
       Xfer2Forms ,
       Ordernumatts,
       ROW_NUMBER()OVER(PARTITION BY UserId,CaseId ORDER BY (SELECT NULL)) rnk
   FROM dbo.Atts
)
DELETE FROM CTE WHERE rnk > 1

Open in new window

0
 

Author Comment

by:amucinobluedot
Comment Utility
Yep, that works. ill assing points to both. Thanks.
0
 

Author Closing Comment

by:amucinobluedot
Comment Utility
Thanks to both.
0
 

Author Comment

by:amucinobluedot
Comment Utility
I tried to create the SP in my Db but I get an error. I tried this:

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]

;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1

Open in new window


Error:
Msg 195, Level 15, State 1, Procedure admin_removeduplicate_links, Line 5 [Batch Start Line 7]
'cte_dups' is not a recognized option.


I am using MS SQL 2008R2
0
 
LVL 17

Expert Comment

by:Pawan Kumar Khowal
Comment Utility
Here it is

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]
AS

BEGIN

	;WITH cte_dups AS (
		SELECT
			AttsId,
			ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
		FROM dbo.Atts
	)
	DELETE FROM cte_dups
	WHERE row_num > 1

END

Open in new window

0
 

Author Comment

by:amucinobluedot
Comment Utility
Thx!
0
 
LVL 69

Expert Comment

by:ScottPletcher
Comment Utility
I think all you really have to have is the word "AS".  I don't add an extra "BEGIN" unless I really need it.

CREATE PROCEDURE [dbo].[admin_removeduplicate_links]
AS
;WITH cte_dups AS (
    SELECT
        AttsId,
        ROW_NUMBER() OVER(PARTITION BY CaseId, UserId ORDER BY AttsId DESC) AS row_num
    FROM dbo.Atts
)
DELETE FROM cte_dups
WHERE row_num > 1
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
Any business that wants to seriously grow needs to keep the needs and desires of an international audience of their websites in mind. Making a website friendly to international users isn’t prohibitively expensive and can provide an incredible return…
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.
HTML5 has deprecated a few of the older ways of showing media as well as offering up a new way to create games and animations. Audio, video, and canvas are just a few of the adjustments made between XHTML and HTML5. As we learned in our last micr…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now