• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1232
  • Last Modified:

Compare multiple folders for file differences

I began to compare 2 folder structures to find files that did not match by date and size, but the requirment has been changed to 4 folders and I am stuck.

So here is what I am trying to do:
We upload several hundred folders\files to 4 different servers.  The files must all match.  Sometimes a file will not copy properly.  So I need a script to read all four directories and compare all the files to make sure they match by size and date.
Output should only be a simple list that shows me the files that didn't match.

Any ideas?
Thanks.

I can do two folders but am lost on four.  Also, this output is confusing.  Not sure how to only list those that don't match.
$path1 = "\\path\folder
$path2 = "\\path\folder1
$dif = Compare-Object -ReferenceObject $path1 -DifferenceObject $path2 -Property FullName, Length, LastWriteTime
$dif | ft -AutoSize

Open in new window

0
greetrufus
Asked:
greetrufus
  • 5
  • 4
  • 2
1 Solution
 
footechCommented:
Pretty much the only way I think this can be done is to run the compare one pair at a time.
The output shows everything that is different, so it is showing only those that don't match.  If there is a file called test.txt that is 1 byte in folder A but is 2 bytes in folder B, it will tell you about the file in each folder.  To make this a little more useful in your case, if folder A is the standard, then we want to see anything in folder A for which there isn't a exact match in folder B.  We can do this by filtering the results and looking at the SideIndicator, which shows whether the difference is from the ReferenceObject (left side) or DifferenceObject (right side).
If we need the full path of the file, we'll need to pass the object and look at it's properties to retrieve it.

$source = "\\sourceserver\folder"
$destinations = "\\server1\folder1", "\\server2\folder2", "\\server3\folder3", "\\server4\folder4"
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

0
 
SubsunCommented:
If the files are copied from One share to other 3 shares and you are trying to find if any files are missing then footech's logic will work. Else if the folders are updated individually then you may have to compare the other folders too. I mean, compare 2nd  folder with 3rd and 4th and 3rd with 4th for difference.
0
 
greetrufusAuthor Commented:
I see the problem.  So if is use one as master and compare the others to master, that should work.  I am trying to read the paths in as parameters but this doesnt' seem to be working.

test.pl -p C:\testing\folder1 -c C:\testing\folder2, C:\testing\folder3

Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
} 
}

Open in new window

0
NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

 
SubsunCommented:
Try this and see.. test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
0
 
greetrufusAuthor Commented:
Runs but empty results.
0
 
footechCommented:
When I posted my code I hadn't closed the scriptblock for the Where cmdlet.  Looks like you may have added a brace at the end.
Here's the code that works.
Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=")} | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

I had assumed that you were already generating the objects to be compared and worked out the compare syntax, so I didn't test or put anything special in for that.  Just to point out that if you're running
test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
all the files should show since the fullname with the path will always be different.  I wonder if we're going to have to work out something different where we compare the source and destination folder by folder, just looking at the name property instead of the fullname.
0
 
greetrufusAuthor Commented:
I came up with this, that will show files that don't match, but it is not showing files that don't exist.
For example: c:\folder1\test.txt does not exist on c:\folder2\ but the missing file test.txt is not showing in the results :
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

$master = Get-ChildItem $PrimaryPath -Recurse  -force
$ComparePath | % { 
   Compare-Object -ReferenceObject $master -DifferenceObject (Get-ChildItem $_ -Recurse | ? { !$_.PSIsContainer }) -Property Name, Length, LastWriteTime -PassThru | where { $_.SideIndicator -eq "=>" } | select fullname 
}

Open in new window

0
 
footechCommented:
If you change the sideindicator to "<=" it should show files that exist in the master that don't show in $comparepath.
0
 
greetrufusAuthor Commented:
Tried that and get very strange results.
Plus, i need the output to be the folder\file that is missing.

Example output:

Comparing c:\folder1 to c:\folder2
 MISSING FILE:  C:\folder2\test.txt

Comparing c:\folder1 to c:\folder3
 MISSING FILE:  C:\folder3\test.txt
0
 
footechCommented:
What results are you seeing?  In my tests I didn't see anything unexpected.  I'll try running your code to see if it points out anything that I'm not thinking of.

Unfortunately Compare-Object won't show objects in that way (it shows objects that are present, not ones that aren't - you just have to infer which ones are missing).  I would say try to find a way to use the output as I pointed out, or we'll need to hack up the output to try to get it into something you can use (for instance - modifying the string that is returned by the fullname property, I think by using a split and replace).
0
 
greetrufusAuthor Commented:
Also, another solution I found that works too
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

#Get filelist with relativepath property
function Get-FilesWithRelativePath ($Path) {
    Get-ChildItem $Path -Recurse | ? { !$_.PSIsContainer } | % { 
        Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.Substring($Path.Length)
        $_
    }
}

#If path exists and is folder
if (Test-Path $PrimaryPath -PathType Container) {
    #Get master fileslist
    $Masterfiles = Get-FilesWithRelativePath (Resolve-Path $PrimaryPath).Path

    #Compare folders
    foreach ($Folder in $ComparePath) {
        if (Test-Path $Folder -PathType Container) {
            #Getting filelist and adding relative-path property to files
            $ResolvedFolder = (Resolve-Path $Folder).Path
            $Files = Get-FilesWithRelativePath $ResolvedFolder

            #Compare and output filepath to missing or old file
            Compare-Object -ReferenceObject $Masterfiles -DifferenceObject $Files -Property RelativePath, Length, LastWriteTime | ? { $_.SideIndicator -eq "<=" } | Select @{n="FilePath";e={Join-Path $ResolvedFolder $_.RelativePath}}
        } else { Write-Error "$Folder is not a valid foldername. Foldertype: Compare" }
    }

} else { Write-Error "$PrimaryPath is not a valid foldername. Foldertype: Master" }

Open in new window

0

Featured Post

Granular recovery for Microsoft Exchange

With Veeam Explorer for Microsoft Exchange you can choose the Exchange Servers and restore points you’re interested in, and Veeam Explorer will present the contents of those mailbox stores for browsing, searching and exporting.

  • 5
  • 4
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now