Solved

Delete a large list of files

Posted on 2012-03-27
6
635 Views
Last Modified: 2012-04-16
Hi All,

I'm trying to delete bad file archive stubs so they can be restored from backup. I have a 80 meg list of 800,000 files with full local path I need deleted. It works, just need it to be smoother, logging, no errors etc.

Ideally I want to get-content from a txt file then remove-item with -LiteralPath switch as . Obviously avoiding errors escape characters and spaces in directory names as well, and any logging of errors would alos be great though I know this may not be possible.

Originally we have a powershell simple script that looked like this.

get-content "c:\deletelist.txt" | Remove-Item

This works well with a few files that don't have [1] in file name or spaces or escape keys, but using this command with the -LiteralPath option to sidestep this seems impossible to my feeble mind.



Can anyone suggest a way they would do this? Much appreciation in advance.
0
Comment
Question by:MC2010
  • 3
  • 2
6 Comments
 
LVL 13

Expert Comment

by:Yagya Shree
ID: 37774620
Please look into this codeplex solution

http://bulkfilemanager.codeplex.com/
0
 
LVL 8

Expert Comment

by:Brent Challis
ID: 37774953
A PowerShell soultion, with simple logging follows.  Here is a link to a more complex logging process
http://powershell.com/cs/media/p/10676.aspx


function Log-Message
{
    Param
    (
        $LogFileName,
        $Message
    )
    ("{0} {1}" -f (Get-Date), $Message) | Out-File $LogFileName -append
}

$logFile = "c:\DeleteFiles\log.txt"
$files = get-content "C:\DeleteFiles\deletelist.txt"
$count = 0
Log-Message $logFile "Starting deletions"
foreach ($file in $files)
{
    if (Test-Path $file)
    {
        try
        {
            Remove-Item -LiteralPath $file
            Log-Message log.txt "Deleted $file"
            $count++
        }
        catch
        {
            Log-Message $logFile ("There was a problem deleting {0}: {1}" -f $file, $_.Message )
        }
    }
    else
    {
        Log-Message $logFile "File $file not found"
    }
}

Log-Message $logFile ("Completed deleting {0} files" -f $count)
0
 
LVL 1

Author Comment

by:MC2010
ID: 37779659
bchallis : While this is a brilliant script, it still does error out on filenames containing [1] eg doc1[1].doc and reports "cannot find file" this should be fixed by using  -LiteralPath, but as my simple script. it doesn't seem to work.
0
Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

 
LVL 8

Expert Comment

by:Brent Challis
ID: 37779759
I changed to Test-Path line to
    if (Test-Path -LiteralPath $file)

and it worked when I tested it.  Can you try this?

Cheers.
0
 
LVL 8

Accepted Solution

by:
Brent Challis earned 500 total points
ID: 37779858
I just noticed that I missed a modification to my code to use a variable for the file name of the logfile so I am included the updated version here -  with the test for the files existence included.  Also if you want more sophisticated logging than the simple Log-Message function i included, here is a link to some loggin functions that I have written:
http://www.ddls.net.au/blog/2012/03/logging-in-powershell/

function Log-Message
{
    Param
    (
        $LogFileName,
        $Message
    )
    ("[{0}] {1}" -f (Get-Date), $Message) | Out-File $LogFileName -append
}

$logFile = "c:\DeleteFiles\log.txt"
$files = get-content "C:\DeleteFiles\deletelist.txt"
$count = 0
Log-Message $logFile "Starting deletions"
foreach ($file in $files)
{
    if (Test-Path -LiteralPath $file)
    {
        try
        {
            Remove-Item -LiteralPath $file
            Log-Message $logFile "Deleted $file"
            $count++
        }
        catch
        {
            Log-Message $logFile ("There was a problem deleting {0}: {1}" -f $file, $_.Message )
        }
    }
    else
    {
        Log-Message $logFile "File $file not found"
    }
}

Log-Message $logFile ("Completed deleting {0} files" -f $count)
0
 
LVL 1

Author Closing Comment

by:MC2010
ID: 37854127
Excellent! thank you very much for your experience here. It has cured my problem and saved us alot of problems. Well done.
0

Featured Post

Backup Solution for AWS

Read about how CloudBerry Backup fully integrates your backups with Amazon S3 and Amazon Glacier to provide military-grade encryption and dramatically cut storage costs on any platform.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

A brief introduction to what I consider to be the best editor for PowerShell.
Windows 10 came with  a lot of built in applications, Some organisations leave them there, some will control them using GPO's. This Article is useful for those who do not want to have any applications in their image (example:me).
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
In this video, we discuss why the need for additional vertical screen space has become more important in recent years, namely, due to the transition in the marketplace of 4x3 computer screens to 16x9 and 16x10 screens (so-called widescreen format). …

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question