Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

Professional Opinions
Ask a Question
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

troubleshooting Question

Python - Deduplicating and Capturing Duplicates

Avatar of bsumariwalla
bsumariwalla asked on
7 Comments1 Solution215 ViewsLast Modified:

I'm using Python 2.7 and I have a lists of lists with lists.  I'm trying to deduplicate the list of lists and also return every duplication.  How can I do this?  I seem to have code that deduplicates okay, but I can figure out the second part.  Consider the following list

[["apple", "red"]
["apple", "red"]
["apple", "red"]
["apple", "green"]
["banana", "yellow"]]

I'm trying to return
[["apple", "red"]
["apple", "green"]
["banana", "yellow"]]

[["apple", "red"]
["apple", "red"]]

def csv_deduplicate2(csv_list):
    sorted_by_date = sorted(csv_list, key=lambda row: row[2], reverse=True)
    unique_csv_list = []
    duplicate_csv_list = []
    for sorted_row in sorted_by_date:
        if sorted_row not in unique_csv_list:
            unique_csv_list.append(sorted_row) # Produces a unique list
    for sorted_row in sorted_by_date:
        if sorted_row in unique_csv_list:
            duplicate_csv_list.append(sorted_row) # Produces a list of everything, not just duplicates.
    return unique_csv_list, duplicate_csv_list
Avatar of aikimark
aikimarkFlag of United States of America imageGet vaccinated; Social distance; Wear a mask
This problem has been solved!
Unlock 1 Answer and 7 Comments.
See Answers