Dynamic Duplicate Finder Formula for Excel

Posted on 2014-08-28
Last Modified: 2014-08-30
I have a huge spreadsheet of names exported from the contacts of a management system.  Within the management system a particular client could have multiple contacts associated with it.  For instance an account might have a contact for a husband, a wife, and two children, totaling four contacts.  When exported, each of the four contacts would have the same "LookupCode" or "Account Code" in the first column.  

For reasons I won't go into, we had a lot of contacts duplicated within our system.  In the example above, perhaps the contact for the father, I'll use "Bob" as his first name, was duplicated, so the total contacts in that account are 5, two "Bobs," one wife and two children.  Only the one duplicate "Bob" is wrong.  And if "Bob" occurs under 50 other accounts, those aren't duplicates.

I want to export all of our contacts to a spreadsheet and mark the contacts which are duplicate contacts.  In my exported spreadsheet I want a formula in a column titled "Duplicate" which returns "No" if it's not a duplicate within the Account "LookupCode" and "Duplicate" if it is:

LookupCode        FirstName    Duplicate      Why?
CodeA                Bob                  No              1st instance of Bob with CodeA
CodeB                Bob                  No              1st instance of Bob with CodeB
CodeB                Bob               Duplicate      2nd instance of Bob with CodeB

I've used a formula to eliminate duplicates in a list before which looks like:


But for my needs the formula can't look for Bob's in the whole of column B, but only as far up as the first instance of the code in column A.  Makes sense?  

I've attached a spreadsheet showing the real thing.
Question by:witzph1
    LVL 50

    Assisted Solution




    It produces the correct result for your data sample.

    Edit: Another option is to ensure the range for the Countif() starts with the first row of  the current LookupCode and extends to the current row. You can do that with


    The first index function INDEX(B:B,MATCH(A2,A:A,0)) finds the row with the first occurrence of the lookup code in the current row and returns the cell in column B of that row. The second index function returns the cell column B in the current row. The range operator ":" combines the two cells into a range that the Countif function can use.  

    Step through the formula with the Formula Evaluation tool to see how the formulas get resolved into ranges. For example in row 18 of your sample file.

    cheers, teylyn
    LVL 7

    Accepted Solution

    What if you concatenated LookUp Code and First Name in a new column and then used =IF(COUNTIF($B$2:Bx,Bx)>1,"Duplicate","No") on the new column?

    Author Comment

    Teylyn, you first and more simple formula worked, but only if the LookupCodes were in exact order.  They would be in order almost always.  But your second, more complex formula worked no matter what the order.  

    That second formula is a work of art!!  I looked and looked at it, using the formula helper, the evaluator, etc.  I've used match and index before, but the combo of everything takes a bit of thinking to wrap your head around what exactly it is doing.  I'm impressed.

    Tomfarrar, your idea actually worked perfectly also.  Honestly, I tend towards simplicity and things I can understand, so I am leaning towards your idea.  I know how to cancatonate and putting a formula to cancatonate in the top field and filling down with a double-click, then entering the formula I had proposed and was familiar with is very comfortable.

    I'm going to split this one between the two of you since they are both great answers.  I truly appreciate both of your help!! Thank you so much and have a great weekend.

    Author Closing Comment

    Thank you so much!
    LVL 7

    Expert Comment

    Glad to help.

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    A little background as to how I came to I design this code: Around 5 years ago I designed an add-in that formatted Excel files to a corporate standard, applying different cell colours and font type depending on whether the cells contained inputs,…
    Outlook Free & Paid Tools
    This Micro Tutorial demonstrates how to create Excel charts: column, area, line, bar, and scatter charts. Formatting tips are provided as well.
    This Micro Tutorial demonstrate the bugs in Microsoft Excel for Mac with Pivot Charts.

    730 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    15 Experts available now in Live!

    Get 1:1 Help Now