To delete million of temp files from production environment

contactsam
contactsam used Ask the Experts™
on
Hello,

We have requirement to delete million of temp attachments from  windows server to ease up the storage space.
This attachment had been since in existence since 2014 - until as of date.

Can you please advise on the following points

a) To begin with we will be interested in going with 6 months delete from 2014 onwards. Once the six month
   delete is complete from 1-1-2014 - 6-30-2014, then afterwards we will proceed with next 6 months and so on.
   Approx count will be around 3 to 4 millions for every 6 months.
   We will be interested to proceed with phase delete instead of big bang delete as this is production environment.

b) Can you provide if there are any tools or command available to perform the above requirement.


Regards
Tom
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Commented:
You could use multiple methods but probably the easiest will be with powershell:
$start = New-Object DateTime 2014, 1, 1
$end = $now.AddMonths(6)
Get-ChildItem -Path "c:\_admin\temptest\" -Include "*" -Recurse | Where {$_.LastAccessTime -le $end -and $_.LastAccessTime -ge $now} | Remove-Item -WhatIf

Open in new window

-saige-

Author

Commented:
Hello Saige,

Thank you for the update.
Question on the command that you shared
Get-ChildItem -Path "c:\_admin\temptest\" -Include "*" -Recurse | Where {$_.LastAccessTime -le $end -and $_.LastAccessTime -ge $now} | Remove-Item -WhatIf

Can we put something "c:\siebelfs\S_EAI*.SAF" in the above command
Is -Recurse mandatory, in our scenario there are no deep subfolders that needs to be deleted.

 Before we run the command to remove, can i modify the script to list the S_EAI*.SAF" files to validate the total count before deleting the files for 6 months.

Regards
Tom

Commented:
You mean like this -
Get-ChildItem -Path "c:\siebelfs\" -Include "S_EAI*.SAF" | Where {$_.LastAccessTime -le $end -and $_.LastAccessTime -ge $now} | Remove-Item -WhatIf

Open in new window

-saige-
Become a Microsoft Certified Solutions Expert

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

Commented:
It would remove.  If you want it to display the items, just remove everything after the where clause; e.g. -
Get-ChildItem -Path "C:\_admin\temptest\r*.xml" | Where {$_.LastWriteTime -le $end -and $_.LastWriteTime -ge $start}

Open in new window

Produces the following output -
    Directory: C:\_admin\temptest


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        2/15/2019  10:15 AM          12623 request.xml
-a----         4/9/2019  10:15 AM          90371 request2.xml
-a----         4/9/2019   2:03 PM          10777 request3.xml
-a----        4/11/2019  10:35 AM          10629 request4.xml
-a----        4/11/2019  10:40 AM          40988 request5.xml
-a----        4/15/2019   2:08 PM          16910 request6.xml
-a----         5/8/2019   2:30 PM           1362 request7.xml
-a----        5/24/2019   2:47 PM          11180 request8.xml

Open in new window

-saige-

Commented:
You could also redirect the output of the remove item to a file:
Get-ChildItem "c:\siebelfs\S_EAI*.SAF" -Force | 
Where {$_.LastWriteTime -le $end -and $_.LastWriteTime -ge $start} |
ForEach-Object {
    $file = $_
    try {
        Remove-Item $file.fullname -ErrorAction Stop
        Add-Content "c:\siebelfs\remove-item.log" "`nDeleting file: $file"
    } catch {
        Add-Content "c:\siebelfs\remove-item.log" "`nError deleting file: $file, $_"
    }
}

Open in new window

-saige-

Author

Commented:
Hello Saige

After running the script, ran into issue, you cannot call a method on null value expression,
screenshot attached with this update.
The script is exactly the same which you shared to list all the files of 2019
$start = New-Object DateTime 2019, 1, 1
$end = $now.AddMonths(6)
Get-ChildItem -Path "c:\siebelfs\" -Include "S_EAI*.SAF"

Objective is to get the count in the first part and second part is to delete the S_EAI*.SAF

Regards
Tom

Commented:
Change to the following:
$start = New-Object DateTime 2019, 1, 1
$end = $start.AddMonths(6)
Get-ChildItem "c:\siebelfs\S_EAI*.SAF"

Open in new window

-saige-

Author

Commented:
Thanks, the script worked perfect, unfortunately i didnt get the listing of 6 months. Do I need to redirect this to output file

Commented:
Screen shot is in the wrong format.

-saige-

Commented:
To get the listing of the six months, you have to include the where clause...  You don't have to output to file.  Corrected script including where clause is:
$start = New-Object DateTime 2019, 1, 1
$end = $start.AddMonths(6)
Get-ChildItem "c:\siebelfs\S_EAI*.SAF" | Where {$_.LastWriteTime -le $end -and $_.LastWriteTime -ge $start}

Open in new window

-saige-

Commented:
When you are ready to remove the items, simply change your script to this -
$start = New-Object DateTime 2019, 1, 1
$end = $start.AddMonths(6)
Get-ChildItem "c:\siebelfs\S_EAI*.SAF" | Where {$_.LastWriteTime -le $end -and $_.LastWriteTime -ge $start} | Remove-Item

Open in new window

-saige-

Author

Commented:
Perfect and to get the total count of the files, should the script need to be modified
Finally, delete the files during that period, this you had already shared
Commented:
To get the total count:
Get-ChildItem "c:\_admin\temptest\r*.xml" -Force | Where {$_.LastWriteTime -le $end -and $_.LastWriteTime -ge $start} | ForEach-Object { Remove-Item $_; Write-Output $_ } | Measure-Object

Open in new window

Produces the following results -Capture.PNG-saige-

Author

Commented:
Thank you Saige, Above script helped and it worked to perfection.
Now we will be testing this script against millions of temp attachment.
I may touch base if need arises .

Appreciate for your help

Regards
Tom

Commented:
Glad to be of assistance...

-saige-

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial