[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now


duplicate files tool

Posted on 2014-08-08
Medium Priority
Last Modified: 2014-08-08
I need a software to scan a directory of choice, list all duplicates in all sub folders, then delete all duplicate copies, and keep one copy, ideally if the report it produces was in alphabetic order and keeps only the first copy, i.e if

pma111.docx was found in both

afolder and b folder and c folder

then keep the copy in afolder and delete the copy in bfolder, cfolder etc. Ideally a free tool.

I found one tool which locates the duplicates fine, but then you have to manually delete the duplicates which takes hours
Question by:pma111
LVL 23

Assisted Solution

by:Thomas Grassi
Thomas Grassi earned 668 total points
ID: 40248334
I think it would be better to write yourself a script to do this for you.

Here is one I found for you that will list all the duplicates

In the routine you can add the process to delete the files as you need.

check this out


Assisted Solution

perolin earned 668 total points
ID: 40248336
Use Windows Server 2012 - there Deduplication is as feature/roll available

Author Comment

ID: 40248395
I cant just purchase a copy of Windows server 2012, I need something free
LVL 11

Accepted Solution

Joseph O'Loughlin earned 664 total points
ID: 40248498
I have found Clonespy to be a powerful duplicate remover
Take the time to explore the different options.  For example from what you say, I believe the option you want is to preserve the files with the shortest path.
LVL 39

Expert Comment

ID: 40248538
Deleting files with duplicate file names using a scripted method is fine if you are absolutely sure that you aren't accidentally deleting different file versions with the same names.  Doing a binary comparison is a different ball game and you tend to find that you have to pay for most of the better duplicate file finders that do a proper comparison.

What criterion/criteria are you currently using to determine whether files are duplicates?

File name only
File name and file size
File name, file size, and creation/last modified date
A file hash (MD5, SHA1, CRC32, SHA-256/512/384)

Personally I use software named Beyond Compare for comparing files and folders:
"Pricing starts at $30 for a standard, single user, single platform license."

You can set this software to show differences only, matches only, or show all and determine the match or mismatch by colour code in the side-by-side panes.  It still involves an element of manual deletion, but if filtered to show matches only you can do this very quickly.

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is about my experience upgrading my consulting machine to Windows 10 Version 1709 (The Fall 2017 Creator Update)
Mailbox Corruption is a nightmare every Exchange DBA wishes he never has. Recovering from it can be super-hectic if not entirely futile. And though techniques like the New-MailboxRepairRequest cmdlet have been designed to help with fixing minor corr…
This video Micro Tutorial shows how to password-protect PDF files with free software. Many software products can do this, such as Adobe Acrobat (but not Adobe Reader), Nuance PaperPort, and Nuance Power PDF, but they are not free products. This vide…
Please read the paragraph below before following the instructions in the video — there are important caveats in the paragraph that I did not mention in the video. If your PaperPort 12 or PaperPort 14 is failing to start, or crashing, or hanging, …
Suggested Courses

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question