Solved

Delete a large list of files

Posted on 2012-03-27
6
639 Views
Last Modified: 2012-04-16
Hi All,

I'm trying to delete bad file archive stubs so they can be restored from backup. I have a 80 meg list of 800,000 files with full local path I need deleted. It works, just need it to be smoother, logging, no errors etc.

Ideally I want to get-content from a txt file then remove-item with -LiteralPath switch as . Obviously avoiding errors escape characters and spaces in directory names as well, and any logging of errors would alos be great though I know this may not be possible.

Originally we have a powershell simple script that looked like this.

get-content "c:\deletelist.txt" | Remove-Item

This works well with a few files that don't have [1] in file name or spaces or escape keys, but using this command with the -LiteralPath option to sidestep this seems impossible to my feeble mind.



Can anyone suggest a way they would do this? Much appreciation in advance.
0
Comment
Question by:MC2010
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 13

Expert Comment

by:Yagya Shree
ID: 37774620
Please look into this codeplex solution

http://bulkfilemanager.codeplex.com/
0
 
LVL 8

Expert Comment

by:Brent Challis
ID: 37774953
A PowerShell soultion, with simple logging follows.  Here is a link to a more complex logging process
http://powershell.com/cs/media/p/10676.aspx


function Log-Message
{
    Param
    (
        $LogFileName,
        $Message
    )
    ("{0} {1}" -f (Get-Date), $Message) | Out-File $LogFileName -append
}

$logFile = "c:\DeleteFiles\log.txt"
$files = get-content "C:\DeleteFiles\deletelist.txt"
$count = 0
Log-Message $logFile "Starting deletions"
foreach ($file in $files)
{
    if (Test-Path $file)
    {
        try
        {
            Remove-Item -LiteralPath $file
            Log-Message log.txt "Deleted $file"
            $count++
        }
        catch
        {
            Log-Message $logFile ("There was a problem deleting {0}: {1}" -f $file, $_.Message )
        }
    }
    else
    {
        Log-Message $logFile "File $file not found"
    }
}

Log-Message $logFile ("Completed deleting {0} files" -f $count)
0
 
LVL 1

Author Comment

by:MC2010
ID: 37779659
bchallis : While this is a brilliant script, it still does error out on filenames containing [1] eg doc1[1].doc and reports "cannot find file" this should be fixed by using  -LiteralPath, but as my simple script. it doesn't seem to work.
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 8

Expert Comment

by:Brent Challis
ID: 37779759
I changed to Test-Path line to
    if (Test-Path -LiteralPath $file)

and it worked when I tested it.  Can you try this?

Cheers.
0
 
LVL 8

Accepted Solution

by:
Brent Challis earned 500 total points
ID: 37779858
I just noticed that I missed a modification to my code to use a variable for the file name of the logfile so I am included the updated version here -  with the test for the files existence included.  Also if you want more sophisticated logging than the simple Log-Message function i included, here is a link to some loggin functions that I have written:
http://www.ddls.net.au/blog/2012/03/logging-in-powershell/

function Log-Message
{
    Param
    (
        $LogFileName,
        $Message
    )
    ("[{0}] {1}" -f (Get-Date), $Message) | Out-File $LogFileName -append
}

$logFile = "c:\DeleteFiles\log.txt"
$files = get-content "C:\DeleteFiles\deletelist.txt"
$count = 0
Log-Message $logFile "Starting deletions"
foreach ($file in $files)
{
    if (Test-Path -LiteralPath $file)
    {
        try
        {
            Remove-Item -LiteralPath $file
            Log-Message $logFile "Deleted $file"
            $count++
        }
        catch
        {
            Log-Message $logFile ("There was a problem deleting {0}: {1}" -f $file, $_.Message )
        }
    }
    else
    {
        Log-Message $logFile "File $file not found"
    }
}

Log-Message $logFile ("Completed deleting {0} files" -f $count)
0
 
LVL 1

Author Closing Comment

by:MC2010
ID: 37854127
Excellent! thank you very much for your experience here. It has cured my problem and saved us alot of problems. Well done.
0

Featured Post

Guide to Performance: Optimization & Monitoring

Nowadays, monitoring is a mixture of tools, systems, and codes—making it a very complex process. And with this complexity, comes variables for failure. Get DZone’s new Guide to Performance to learn how to proactively find these variables and solve them before a disruption occurs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A quick Powershell script I wrote to find old program installations and check versions of a specific file across the network.
A recent project that involved parsing Tableau Desktop and Server log files to extract reusable user queries for use in other systems. I chose to use PowerShell to gather the data, and SharePoint to present it...
In this video, we discuss why the need for additional vertical screen space has become more important in recent years, namely, due to the transition in the marketplace of 4x3 computer screens to 16x9 and 16x10 screens (so-called widescreen format). …
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question