duplicate files tool

I need a software to scan a directory of choice, list all duplicates in all sub folders, then delete all duplicate copies, and keep one copy, ideally if the report it produces was in alphabetic order and keeps only the first copy, i.e if

pma111.docx was found in both

afolder and b folder and c folder

then keep the copy in afolder and delete the copy in bfolder, cfolder etc. Ideally a free tool.

I found one tool which locates the duplicates fine, but then you have to manually delete the duplicates which takes hours
LVL 3
pma111Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Thomas GrassiSystems AdministratorCommented:
I think it would be better to write yourself a script to do this for you.

Here is one I found for you that will list all the duplicates

In the routine you can add the process to delete the files as you need.

check this out

http://blogs.technet.com/b/heyscriptingguy/archive/2008/01/10/how-can-i-list-all-the-duplicate-files-in-a-folder-and-its-subfolders.aspx
0
perolinCommented:
Use Windows Server 2012 - there Deduplication is as feature/roll available
0
pma111Author Commented:
I cant just purchase a copy of Windows server 2012, I need something free
0
Joseph OLoughlinIT Support SpecialistCommented:
I have found Clonespy to be a powerful duplicate remover
http://www.clonespy.com/
Take the time to explore the different options.  For example from what you say, I believe the option you want is to preserve the files with the shortest path.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
BillDLCommented:
Deleting files with duplicate file names using a scripted method is fine if you are absolutely sure that you aren't accidentally deleting different file versions with the same names.  Doing a binary comparison is a different ball game and you tend to find that you have to pay for most of the better duplicate file finders that do a proper comparison.

What criterion/criteria are you currently using to determine whether files are duplicates?

File name only
File name and file size
File name, file size, and creation/last modified date
A file hash (MD5, SHA1, CRC32, SHA-256/512/384)

Personally I use software named Beyond Compare for comparing files and folders:
http://www.scootersoftware.com/moreinfo.php
"Pricing starts at $30 for a standard, single user, single platform license."

You can set this software to show differences only, matches only, or show all and determine the match or mismatch by colour code in the side-by-side panes.  It still involves an element of manual deletion, but if filtered to show matches only you can do this very quickly.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Software

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.