We help IT Professionals succeed at work.
Get Started

Trying to find a way how I can delete duplicates of a csv files.

jay bro
jay bro asked
Last Modified: 2018-02-06

I have a list of files like around 45 000. I am trying to find a way how i can delete the duplicates.
A file would be listed like the following


So there could be duplicate files and I need to find a way how i can remove them and only keep 1 of the files say by the last modified date.

I am looking for a script either can be run on windows or linux ubuntu.

I have tried different file duplicator programs like looking at hash or md5 but it doesn't detect it.
Watch Question
Distinguished Expert 2020
This problem has been solved!
Unlock 1 Answer and 13 Comments.
See Answer
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE