<div>
I doubt the PS approach will help with that, as it still has to traverse all file system info to determine the top n files. But of course you can try.<br />

<pre><code class="code-1 nolinks" id="code-20-41453250-1"><span>Get-ChildItem C:\ -recurse | select FullName, Length | sort Length -Desc | Select -First 50</span>
</code></pre>
</div>


<div>
Thanks...Does this list the size of the file though?
</div>


<div>
Yes, it shows the full path plus the size.
</div>


<div>
yes and add this<br />
| export-csv C:\file_size\log.csv<br />
the you have the output as a file
</div>


<div>
Thanks I am trying it now..
</div>


<div>
the script is running but I am getting messages that the file paths it is hitting are to long..
</div>


<div>
By default the &quot;length&quot; property does not show size in MB.<br />
Some addition to script posted by Qlemo.<br />
will give output to .csv and size will be in MB<br />

<pre><code class="code-1 nolinks" id="code-20-41453337-1"><span>Get-ChildItem C:\ -Recurse | select Name, Directory, @{N=&quot;Size(MB)&quot;;E={[Math]::Round($_.length/1MB,2)}} | sort &quot;Size(MB)&quot; -Descending | select -First 50 | Export-Csv -Path d:\Top50Files.csv -NoTypeInformation</span>
</code></pre>
</div>


<div>
Thanks... I will try this tomorrow and report back.. With this be able to handles files names with very long path names?
</div>


<div>
As the error says, its due to the path of file is more than 256 characters.<br />
You will face the same error by running the script I posted coz there is no change is the cmdlet used (get-childitem).
</div>


<div>
This uses robocopy to produce the file list (but it will <b>not</b> actually copy anything), as robocopy doesn't care about long paths. It should have a pretty low memory footprint, too, since it doesn't collect all items first and then sorts them, but only collects the top n biggest ones.<br />
Can't promise anything concerning speed, though.<br />
It returns an array of PSCustom objects with two properties, FullName and Length. You can process that output any way you feel like, for example like this:<br />

<pre><code class="code-1 nolinks" id="code-20-41453381-1"><span>$Top = .\Whatever.ps1 -Path E:\Wherever</span>
<span>$Top | fl</span>
<span>$Top | Export-Csv -Path C:\Wherever\Top.csv -NoTypeInformation</span>
</code></pre>
<pre><code class="code-10 nolinks" id="code-20-41453381-2"><span>[CmdletBinding()]</span>
<span>Param(</span>
<span>	[string]$Path = $(Get-Location -PSProvider Filesystem),</span>
<span>	[uint32]$Top = 50</span>
<span>)</span>
<span>$List = 1..$Top | ForEach-Object {New-Object -TypeName PSObject -Property @{&quot;Length&quot; = 0}}</span>
<span>&amp; robocopy.exe $Path C:\Dummy_Must_Not_Exist *.* /L /s /nc /njh /njs /ndl /fp /bytes /r:0 | ForEach-Object {</span>
<span>	If ($_) {</span>
<span>		$Split = $_.Split(&quot;`t&quot;)</span>
<span>		$Length = [int]$Split[3].Trim()</span>
<span>		If ($Length -gt $List[0].Length) {</span>
<span>			$List[0] = $Split[4].Trim() | Select-Object -Property @{Name=&quot;FullName&quot;; Expression={$_};}, @{Name=&quot;Length&quot;; Expression={$Length}}</span>
<span>			$List = $List | Sort-Object -Property Length</span>
<span>		}</span>
<span>	}</span>
<span>}</span>
<span>$List | Where-Object {$_.Length -gt 0}</span>
</code></pre>
We'll worry about how (considering the long file paths) to get the rest of the information you want if that runs in a timely fashion.
</div>


<div>
A very clever way to get the top n. Had to think about how it works for some time...
</div>


<div>
Thank for the replied and will look at this more tomorrow but the though of using robocopy even to just list files scare me a bit
</div>


<div>
In the past, I've used successive automaitc SUBST to cut down overly long paths in PowerShell, but that is in no way better (as any advantage) over using RoboCopy. RoboCopy is built-in, so why not use it?
</div>


<div>
A couple of suggestions<br />
<b>Exclude small files</b> -- in this parameter example, 1MB minimum greatly reduced the number of output lines produced by Robocopy in my test. &nbsp;If you know your file system well, you might be able to increase that minimum<br />

<pre><code class="code-1 nolinks" id="code-20-41453788-1"><span>/min:1000000</span>
</code></pre>
<br />
<b>Sort once</b> -- instead of sorting your list every time you add an item, sort all the items and take the top N.
</div>


<div>
compdigit44,<br />
with the /L, argument, robocopy will only list the files - really. All it will do is generate a file list.<br />
<br />
aikimark,<br />
the /min is a nice touch.<br />
The &quot;sort once [all the items]&quot; is exactly what I wanted to avoid. Not knowing anything about the server resources and its file structure, I find it safer to invest a bit of CPU into sorting (not that sorting 50 elements is really that taxing) than to first collect potentially gigabytes of data in RAM.
</div>


<div>
Here's a version that includes the minimum size suggested by aikimark, and more importantly, <b>fixes a sizing issue</b> (sorry); [int] might not be big enough. In addition, it gives you LastWriteTime and size in MB.<br />

<pre><code class="code-10 nolinks" id="code-20-41453892-1"><span>[CmdletBinding()]</span>
<span>Param(</span>
<span>	[string]$Path = $(Get-Location -PSProvider Filesystem),</span>
<span>	[uint32]$Top = 50,</span>
<span>	[string]$MinimumSize = &quot;1MB&quot;</span>
<span>)</span>
<span>$List = 1..$Top | ForEach-Object {New-Object -TypeName PSObject -Property @{&quot;Length&quot; = 0}}</span>
<span>If (($Min = ([int64]1 * $MinimumSize.Replace(&quot; &quot;, &quot;&quot;))) -isnot [int64]) {</span>
<span>	&quot;Unable to parse '$($MinimumSize)' to an integer.&quot; | Write-Error</span>
<span>	Exit 1</span>
<span>}</span>
<span>## Not collecting LastWriteTime yet; it'll be in the same field as the Length, which would require an additional Split().</span>
<span>&amp; robocopy.exe $Path C:\Dummy_Must_Not_Exist *.* /L /s /nc /njh /njs /ndl /fp /bytes /min:$Min /r:0 | ForEach-Object {</span>
<span>	If ($_) {</span>
<span>		$Split = $_.Split(&quot;`t&quot;)</span>
<span>		$Length = [int64]$Split[3].Trim()</span>
<span>		If ($Length -gt $List[0].Length) {</span>
<span>			$List[0] = $Split[4].Trim() | Select-Object -Property @{Name=&quot;FullName&quot;; Expression={$_};}, @{Name=&quot;Length&quot;; Expression={$Length}}</span>
<span>			$List = $List | Sort-Object -Property Length</span>
<span>		}</span>
<span>	}</span>
<span>}</span>
<span>$List | Where-Object {$_.Length -gt 0} | ForEach-Object {</span>
<span>	$Folder = Split-Path -Path $_.FullName -Parent</span>
<span>	$File = Split-Path -Path $_.FullName -Leaf</span>
<span>	$Line = &amp; robocopy.exe $Folder C:\Dummy_Must_Not_Exist $File /L /s /nc /njh /njs /ndl /ns /ts</span>
<span>	$_ | Select-Object -Property `</span>
<span>		*,</span>
<span>		@{Name=&quot;SizeMB&quot;; Expression={&quot;{0:N3}&quot; -f ($_.Length / 1MB)}},</span>
<span>		@{Name=&quot;LastWriteTime&quot;; Expression={[DateTime]($Line.Split(&quot;`t&quot;)[4].Trim())}}</span>
<span>}</span>
</code></pre>
</div>


<div>
Every time you find a file that is larger than the minimum in your list, you are performing the sort.<br />
<br />
You could go through a second round of filtering after the Robocopy <b>/min:###</b> process. &nbsp;<br />
1. Read the results and look for the largest and smallest values. &nbsp;Alternatively, you could do a frequency analysis of the file sizes (a more accurate approach). &nbsp; &nbsp;<br />
2. With the information/data from step 1, you can then filter the Robocopy output to make sorting a simple, non-system-stressing operation.<br />
<br />
===================<br />
For frequency analysis, use a 10x14 array to store the counts of the <b>mantissa digit</b> and <b>exp</b> indexed file counts.<br />
<br />
A simpler frequency analysis can be made of the length of the string numbers, ignoring the mantissa data.<br />
<br />
A simpler approach to the initial filtering would be to start with a large <b>/min:###</b> value and decrease it until you get more than 50 (?N?) files. &nbsp;Save that value for each volume for the next run.<br />
<br />
<b>Example:</b><br />
On my test, I ran the Robocopy against a directory tree with 31000 files. &nbsp;The resulting directory tree output was 4MB. &nbsp;Applying the <b>/min:1000000</b> reduced the file to 30KB with 390 lines.
</div>


<div>
Mark, there only two practical choices:<br />
Either you ignore the amount of data, e.g. because it is of no significance anymore (filtered to something reasonable),<br />
or you sort each time.<br />
<br />
Everything else is sorting theory only. without any practical value for this case. It is making it <u>complex</u>, not simple, to apply statistical methods.<br />
The &quot;sort on change&quot; approach will scale well, as the probability to have the top n values in the list is increasing with each file found, and then no sort is done.<br />
<br />
A sort of 50 objects (with an average of maybe 250 Bytes per entry) should be a very fast operation, without much CPU cost. The more objects to keep, the more cost, that is correct - with O(n^2), if I'm correct.
</div>


<div>
@Q<br />
<br />
O(N * log2(N))<br />
<br />
I did some back-of-the-napkin estimate, based on my earlier test. &nbsp;If I scale up my 8.5GB tree to 17TB, I might expect a concomitant 2000 fold increase in the number of files. &nbsp;So, my 30,900 file problem would scale up to 61,800,000 file problem. &nbsp;Sure, sorting 50 items isn't too terribly expensive, but we don't know the distribution of the files, so we can't eliminate the possibility that we would be doing over 60M sorting operations on our little list.
</div>


<div>
I ran the script below as posted earlier and get a message that states..&quot;Cannot call method on null value expression&quot; yet does should some files.<br />
<br />
Also I noticed the Full name path list a whole bunch of dots at the end. If the script exported to a CSV could we see the full file path? What does the column Length mean? Stupid question how can I be sure that robocopy is not moving any files? Sorry just paranoid...<br />
<br />
<br />

<pre><code class="code-10 nolinks" id="code-20-41455348-1"><span>[CmdletBinding()]</span>
<span>Param(</span>
<span>	[string]$Path = $(Get-Location -PSProvider Filesystem),</span>
<span>	[uint32]$Top = 50,</span>
<span>	[string]$MinimumSize = &quot;1MB&quot;</span>
<span>)</span>
<span>$List = 1..$Top | ForEach-Object {New-Object -TypeName PSObject -Property @{&quot;Length&quot; = 0}}</span>
<span>If (($Min = ([int64]1 * $MinimumSize.Replace(&quot; &quot;, &quot;&quot;))) -isnot [int64]) {</span>
<span>	&quot;Unable to parse '$($MinimumSize)' to an integer.&quot; | Write-Error</span>
<span>	Exit 1</span>
<span>}</span>
<span>## Not collecting LastWriteTime yet; it'll be in the same field as the Length, which would require an additional Split().</span>
<span>&amp; robocopy.exe $Path C:\Dummy_Must_Not_Exist *.* /L /s /nc /njh /njs /ndl /fp /bytes /min:$Min /r:0 | ForEach-Object {</span>
<span>	If ($_) {</span>
<span>		$Split = $_.Split(&quot;`t&quot;)</span>
<span>		$Length = [int64]$Split[3].Trim()</span>
<span>		If ($Length -gt $List[0].Length) {</span>
<span>			$List[0] = $Split[4].Trim() | Select-Object -Property @{Name=&quot;FullName&quot;; Expression={$_};}, @{Name=&quot;Length&quot;; Expression={$Length}}</span>
<span>			$List = $List | Sort-Object -Property Length</span>
<span>		}</span>
<span>	}</span>
<span>}</span>
<span>$List | Where-Object {$_.Length -gt 0} | ForEach-Object {</span>
<span>	$Folder = Split-Path -Path $_.FullName -Parent</span>
<span>	$File = Split-Path -Path $_.FullName -Leaf</span>
<span>	$Line = &amp; robocopy.exe $Folder C:\Dummy_Must_Not_Exist $File /L /s /nc /njh /njs /ndl /ns /ts</span>
<span>	$_ | Select-Object -Property `</span>
<span>		*,</span>
<span>		@{Name=&quot;SizeMB&quot;; Expression={&quot;{0:N3}&quot; -f ($_.Length / 1MB)}},</span>
<span>		@{Name=&quot;LastWriteTime&quot;; Expression={[DateTime]($Line.Split(&quot;`t&quot;)[4].Trim())}}</span>
<span>}</span>
</code></pre>
</div>


<div>
<blockquote>
how can I be sure that robocopy is not moving any files?
</blockquote>
because of the <b>/L</b> command line switch
</div>


<div>
Thanks ... Just wanted to make sure since the Microsoft site use a lower case l and not upper case.. Is the other syntax include just logging?<br />
<br />
what about the error message I am getting or the full file name not being listed? Would it display the file name if exported to CSV... What would need to be changed to export it to a CSV?
</div>


<div>
shouldn't be an issue, since the long path names are coming from Robocopy and not from within PS
</div>


<div>
Stupid question...<br />
<br />
I see you are saving the script to a PS1 file.. for powershell then using the path command.. Isn't this redundant<br />
<br />
\Whatever.ps1 -Path E:\Wherever | Export-Csv -Path C:\Wherever\Top.csv -NoTypeInformation<br />
<br />
Also were do I specific which volume to scan
</div>


<div>
Sorry, I can't quite follow you.<br />
The script accepts three arguments, all optional:<br />
-Path: The path to the folder in which to start the report; default is the current folder (which is not necessarily the script's).<br />
-Top: the number of files to collect in the report; default is 50.<br />
-MinimumSize: the minimum file size in bytes to consider wort checking; this accepts (as you can see in the default value) stuff like &quot;1MB&quot; or &quot;100KB&quot; or &quot;10TB&quot; as well.
</div>


<div>
Thank but how does the script know which drive to scan..
</div>


<div>
With the -Path argument, the same way Get-ChildItem expects the path to list as argument.<br />
Save the script somewhere, for example as C:\Temp\Get-TopFiles.ps1.<br />
Then open a PS console (preferably as Administrator), and if you want to scan for example the folder &quot;E:\BigData&quot;, enter<br />

<pre><code class="code-1 nolinks" id="code-20-41456342-1"><span>$Top = C:\Temp\Get-TopFiles.ps1 -Path E:\BigData</span>
</code></pre>
<b>Do not forget the &quot;$Top =&quot;</b>; it collects the objects returned so that you have more to work with than some console output that will be truncated.<br />
For further processing of that variable and how to export it to csv, check my examples above <a href="http:#a41455680" rel="ugc">http:#a41455680</a>.
</div>


<div>
In a test I ran, the Robocopy output might contain error messages like this:<br />

<pre><code class="code-1 nolinks" id="code-20-41456351-1"><span>2016/02/09 10:18:50 ERROR 5 (0x00000005) Scanning Source Directory c:\users\aikimark\Templates\</span>
<span>Access is denied.</span>
</code></pre>
<br />
So, you will need to check that the file length column you parse is numeric before you add the item to your list.<br />
<br />
Alternatively, you might check that the split line result has more than one item.
</div>


<div>
That's already been taken care of in the latest version. The error lines don't contain tabs, so access to the split array will fail and end up in the Catch.
</div>


<div>
First off I wanted to thank everyone one for their help. I have been testing the script and it is working great you Guy's are Grand Master at this ...<br />
<br />
Right now my goal is to run a scheduled task to scan each of my volume daily overwriting the same CSV name. The after all file are done run a final script to send all CSV's as an email attachment. Is this hard to do?<br />
<br />
&nbsp;Also when I run the script in the CSV what does the LENGTH column represent?
</div>


<div>
<b>Length</b> is the file size (it is the way PowerShell fills the file object). We would not have to keep it, as the script adds <b>SizeMB</b>, but it is no bad idea either to see the &quot;raw&quot; data.<br />
<br />
The &quot;send all CSV&quot; part isn't difficult:<br />

<pre><code class="code-1 nolinks" id="code-20-41458674-1"><span>Send-MailMessage -Subject &quot;Daily Report&quot; -From me@domain.com -To you@domain.com -SmtpServer mail.domain.com -Attachments (get-childitem c:\temp\ -include *.csv | select -Expand FullName)</span>
</code></pre>
or<br />

<pre><code class="code-1 nolinks" id="code-20-41458674-2"><span>get-childitem c:\temp\ -include *.csv | select -Expand Fullname | Send-MailMessage  -Subject &quot;Daily Report&quot; -From me@domain.com -To you@domain.com -SmtpServer mail.domain.com</span>
</code></pre>
</div>


<div>
<div class="content wysiwyg-content">
I have a very large File Server environment with 17TB of data and our current storage monitor solution is not able to scan the volume fast enough to produce a report daily..<br />
<br />
I am not a powershell export but I am looking to see if a scritp would be able to list the Top 50 largest files per volume from largest to smallest, its full path and date of late access , owner and of course size. If it could be exported to a csv this would be great...
</div>
</div>


I have a very large File Server environment with 17TB of data and our current storage monitor solution is not able to scan the volume fast enough to produce a report daily..

I am not a powershell export but I am looking to see if a scritp would be able to list the Top 50 largest files per volume from largest to smallest, its full path and date of late access , owner and of course size. If it could be exported to a csv this would be great...

Powershell Script List Top 50 Largest Files

Windows PowerShell is a task automation and configuration management framework from Microsoft, consisting of a command-line shell and associated scripting language built on the .NET Framework. PowerShell provides full access to the Component Object Model (COM) and Windows Management Instrumentation (WMI), enabling administrators to perform administrative tasks on both local and remote Windows systems as well as WS-Management and Common Information Model (CIM) enabling management of remote Linux systems and network devices.

Powershell

Windows Server 2012 is the server version of Windows 8 and the successor to Windows Server 2008 R2. Windows Server 2012 is the first version of Windows Server to have no support for Itanium-based computers since Windows NT 4.0. Windows Server 2012, now in its second release (Windows Server 2012 Release 2) includes Foundation, Essentials, Standard and Datacenter, and does not support IA-32 or IA-64 processors.