We help IT Professionals succeed at work.

Finding dulplicate records

latzo2
latzo2 asked
on
214 Views
Last Modified: 2011-10-03
I have the following #Temptable and would like to find out if there are duplicate records based on three fields

field 1, 2, 4

field1 | field2 | field3 | filed4 |Rule
1          1          2          2
1          1          2          3
1          1          2          4
1          1          2          4
 
If there are duplicate records then the 'rule' field gets a '1' else '0'

Results

field1 | field2 | field3 | filed4  |Rule
1          1             2        2         0
1          1             2        3         0
1          1             2        4         1
1          1             2        4         1

thanks,
 lat
Comment
Watch Question

CERTIFIED EXPERT
Top Expert 2011

Commented:
update table
  set rule = case when K > 1 then 1 else 0 end
  from table as a inner join
    (select field1,field2,field4 ,count(*) as K from table group by field1,field2,field4)  as b  
   on a.field1=b.field1 and a.field2=b.field2 and a.field4=b.field4



the above will work as long as none of field1,2,4 are null

   
CERTIFIED EXPERT

Commented:
If nulls are possible then just do this on lowfatspread's query:

IsNull(a.field1, 0) =IsNull(b.field1,0) and IsNull(a.field2, 0)=IsNull(b.field2,0) and IsNull(a.field4, 0) = IsNull( b.field4,0)
CERTIFIED EXPERT
Top Expert 2011

Commented:
thanks namasi,
i think it s time for me to get some sleep
;-)
CERTIFIED EXPERT

Commented:
Lowfatspread,

Isn't not too early to get some sleep.?

regards-
CERTIFIED EXPERT
Top Expert 2011

Commented:
2am is kind of late for me...
CERTIFIED EXPERT

Commented:
I live in Canada. It is only 9 PM here.. Have a good night sleep...
Scott PletcherSenior DBA
CERTIFIED EXPERT
Most Valuable Expert 2018
Distinguished Expert 2019

Commented:
Sometimes the EXISTS method will perform better (probably peforms better less often overall than Lowfat's method, but it's still a good technique to know :-) ):


UPDATE #temptable
SET [rule] = CASE WHEN EXISTS(
    SELECT 1
    FROM #temptable temp2
    WHERE #temptable.field1 = temp2.field1
    AND #temptable.field2 = temp2.field2
    AND #temptable.field4 = temp2.field4) THEN 1 ELSE 0 END


Same as above for dealing with NULL(s), if any.

Author

Commented:
hello  Lowfatspread, i think your query works but i';m running it on about 5k records, and so it may take a little bit of time to verify if the results are correct.  can you explain how the syntax works, i'm new to sql this and would like to learn what it means.

thanks,
 lat
CERTIFIED EXPERT
Top Expert 2011
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Thanks for all of your help!

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.