duplicate files tool

Posted on 2014-08-08
Last Modified: 2014-08-08
I need a software to scan a directory of choice, list all duplicates in all sub folders, then delete all duplicate copies, and keep one copy, ideally if the report it produces was in alphabetic order and keeps only the first copy, i.e if

pma111.docx was found in both

afolder and b folder and c folder

then keep the copy in afolder and delete the copy in bfolder, cfolder etc. Ideally a free tool.

I found one tool which locates the duplicates fine, but then you have to manually delete the duplicates which takes hours
Question by:pma111
    LVL 23

    Assisted Solution

    by:Thomas Grassi
    I think it would be better to write yourself a script to do this for you.

    Here is one I found for you that will list all the duplicates

    In the routine you can add the process to delete the files as you need.

    check this out
    LVL 2

    Assisted Solution

    Use Windows Server 2012 - there Deduplication is as feature/roll available
    LVL 3

    Author Comment

    I cant just purchase a copy of Windows server 2012, I need something free
    LVL 11

    Accepted Solution

    I have found Clonespy to be a powerful duplicate remover
    Take the time to explore the different options.  For example from what you say, I believe the option you want is to preserve the files with the shortest path.
    LVL 38

    Expert Comment

    Deleting files with duplicate file names using a scripted method is fine if you are absolutely sure that you aren't accidentally deleting different file versions with the same names.  Doing a binary comparison is a different ball game and you tend to find that you have to pay for most of the better duplicate file finders that do a proper comparison.

    What criterion/criteria are you currently using to determine whether files are duplicates?

    File name only
    File name and file size
    File name, file size, and creation/last modified date
    A file hash (MD5, SHA1, CRC32, SHA-256/512/384)

    Personally I use software named Beyond Compare for comparing files and folders:
    "Pricing starts at $30 for a standard, single user, single platform license."

    You can set this software to show differences only, matches only, or show all and determine the match or mismatch by colour code in the side-by-side panes.  It still involves an element of manual deletion, but if filtered to show matches only you can do this very quickly.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Enabling OSINT in Activity Based Intelligence

    Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

    Workplace bullying has increased with the use of email and social media. Retain evidence of this with email archiving to protect your employees.
    The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
    Viewers will learn how to use the Hootsuite Dashboard.
    After watching the Introduction to GIMP this tutorial will show you additional tools to use in GIMP.

    779 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    13 Experts available now in Live!

    Get 1:1 Help Now