Solved

Problem deleting duplicates using a CTE when there are table joins

Posted on 2010-11-10
10
1,178 Views
Last Modified: 2012-05-10
I've used CTEs to remove duplicates successfully before, but this time round I wanted to delete them from one table having joined to other tables and it comes back with the error:

View or function 'dupnotesCTE' is not updatable because the modification affects multiple base tables. See the attached code for my sql statement.

Not sure where I've gone wrong here...can someone help please?
;WITH dupnotesCTE([FileNo]

      ,[CommentDate]

      ,[CommentTime]

      ,[CommentUser]

      ,[Cat]

      ,[Pri]

      ,[Line]

      ,[Note], Ranking)

AS

(

SELECT	p.[FileNo]

      ,p.[CommentDate]

      ,p.[CommentTime]

      ,p.[CommentUser]

      ,p.[Cat]

      ,p.[Pri]

      ,p.[Line]

      ,p.[Note]

	,Ranking = row_number() OVER(PARTITION BY p.[FileNo]

      ,p.[CommentDate]

      ,p.[CommentTime]

      ,p.[CommentUser]

      ,p.[Cat]

      ,p.[Pri]

      ,p.[Line]

      ,p.[Note] ORDER BY p.[FileNo]

      ,p.[CommentDate]

      ,p.[CommentTime]

      ,p.[CommentUser]

      ,p.[Cat]

      ,p.[Pri]

      ,p.[Line]

      ,p.[Note])

FROM filenotes p

join detailtable d on ltrim(rtrim(p.FileNo)) = ltrim(rtrim(d.FileNo))

join country c on d.country_no = c.country_no

where c.continent in ('Asia','Europe')

)



delete from dupnotesCTE

where ranking > 1

Open in new window

0
Comment
Question by:Grffster
  • 5
  • 5
10 Comments
 
LVL 32

Expert Comment

by:Erick37
ID: 34101354


...

delete p
FROM filenotes p
JOIN dupnotesCTE c ON c.FileNo = p.fileNo --??
where c.ranking > 1
0
 

Author Comment

by:Grffster
ID: 34101830
This won't work though will it as, say you have a pair of duplicates A and B. You have A in dupnotesCTE with rank 1, and B in dupnotesCTE with rank 2. When you link from filenotes to dupnotes you'll have 4 records:

filenotes.A and dupnotes.A (rank 1)
filenotes.A and dupnotes.B (rank 2)
filenotes.B and dupnotes.A (rank 1)
filenotes.B and dupnotes.B (rank 2)

Your "where c.ranking > 1" will remove the 2nd and 4th combinations but you're left with two records i.e. combinations 1 and 3 above.

May be mistaken, but my query seems to reflect this.
0
 
LVL 32

Expert Comment

by:Erick37
ID: 34102171
Does filenotes table have an identity column or primary key that you could use to join the cte?

0
 

Author Comment

by:Grffster
ID: 34102188
No it doesn't sorry.
0
 
LVL 32

Expert Comment

by:Erick37
ID: 34102227
So all of the columns with the duplicate FileNo are also duplicate?
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:Grffster
ID: 34102264
Yes, all the fields per fileno duplicate are identical i.e. a true duplicated record with the same data in each column:

[FileNo],[CommentDate],[CommentTime],[CommentUser],[Cat],[Pri],[Line],[Note]
0
 
LVL 32

Expert Comment

by:Erick37
ID: 34102504
Having an identity column would greatly simplify identifying the duplicates.

One possibility would be to copy the records over to a temp table with an identity column.
Remove the duplicates from the temp table.
Delete all the records from the user table that you originally copied to the temp table.
Insert the cleaned records back into filenotes table.

Here are some other alternatives:
"How to delete duplicate records or rows among identical rows in a table where no primary key exists"
http://www.kodyaz.com/articles/delete-duplicate-records-rows-in-a-table.aspx
0
 

Author Comment

by:Grffster
ID: 34102929
I've added an identity column to the filenotes table now. How should it look now? Do I do something like at the bottom of your last link in method 3?
0
 
LVL 32

Accepted Solution

by:
Erick37 earned 500 total points
ID: 34103157
You will add the identity column to the CTE and use that to join the table to the cte.

;WITH dupnotesCTE AS
(
  SELECT
      IdentityColumn
      ,[FileNo]
      ...
      ,row_number() OVER(PARTITION BY p.[FileNo] order by IdentityColumn) AS Ranking
     ...

)
delete p
FROM filenotes p
JOIN dupnotesCTE c ON c.IdentityColumn = p.IdentityColumn AND c.ranking > 1
0
 

Author Closing Comment

by:Grffster
ID: 34103515
That's worked a dream thanks Eric.
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
OPENDATASOURCE 8 31
How to enforce inte 8 32
SQL Encryption question 2 42
Grid querry results 41 56
In this article I will describe the Backup & Restore method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now