Solved

Compare multiple folders for file differences

Posted on 2013-01-21
11
1,074 Views
Last Modified: 2013-01-24
I began to compare 2 folder structures to find files that did not match by date and size, but the requirment has been changed to 4 folders and I am stuck.

So here is what I am trying to do:
We upload several hundred folders\files to 4 different servers.  The files must all match.  Sometimes a file will not copy properly.  So I need a script to read all four directories and compare all the files to make sure they match by size and date.
Output should only be a simple list that shows me the files that didn't match.

Any ideas?
Thanks.

I can do two folders but am lost on four.  Also, this output is confusing.  Not sure how to only list those that don't match.
$path1 = "\\path\folder
$path2 = "\\path\folder1
$dif = Compare-Object -ReferenceObject $path1 -DifferenceObject $path2 -Property FullName, Length, LastWriteTime
$dif | ft -AutoSize

Open in new window

0
Comment
Question by:greetrufus
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 2
11 Comments
 
LVL 40

Expert Comment

by:footech
ID: 38803839
Pretty much the only way I think this can be done is to run the compare one pair at a time.
The output shows everything that is different, so it is showing only those that don't match.  If there is a file called test.txt that is 1 byte in folder A but is 2 bytes in folder B, it will tell you about the file in each folder.  To make this a little more useful in your case, if folder A is the standard, then we want to see anything in folder A for which there isn't a exact match in folder B.  We can do this by filtering the results and looking at the SideIndicator, which shows whether the difference is from the ReferenceObject (left side) or DifferenceObject (right side).
If we need the full path of the file, we'll need to pass the object and look at it's properties to retrieve it.

$source = "\\sourceserver\folder"
$destinations = "\\server1\folder1", "\\server2\folder2", "\\server3\folder3", "\\server4\folder4"
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

0
 
LVL 40

Expert Comment

by:Subsun
ID: 38804558
If the files are copied from One share to other 3 shares and you are trying to find if any files are missing then footech's logic will work. Else if the folders are updated individually then you may have to compare the other folders too. I mean, compare 2nd  folder with 3rd and 4th and 3rd with 4th for difference.
0
 

Author Comment

by:greetrufus
ID: 38806027
I see the problem.  So if is use one as master and compare the others to master, that should work.  I am trying to read the paths in as parameters but this doesnt' seem to be working.

test.pl -p C:\testing\folder1 -c C:\testing\folder2, C:\testing\folder3

Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
} 
}

Open in new window

0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 40

Expert Comment

by:Subsun
ID: 38806060
Try this and see.. test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
0
 

Author Comment

by:greetrufus
ID: 38806080
Runs but empty results.
0
 
LVL 40

Accepted Solution

by:
footech earned 500 total points
ID: 38807031
When I posted my code I hadn't closed the scriptblock for the Where cmdlet.  Looks like you may have added a brace at the end.
Here's the code that works.
Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=")} | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

I had assumed that you were already generating the objects to be compared and worked out the compare syntax, so I didn't test or put anything special in for that.  Just to point out that if you're running
test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
all the files should show since the fullname with the path will always be different.  I wonder if we're going to have to work out something different where we compare the source and destination folder by folder, just looking at the name property instead of the fullname.
0
 

Author Comment

by:greetrufus
ID: 38811919
I came up with this, that will show files that don't match, but it is not showing files that don't exist.
For example: c:\folder1\test.txt does not exist on c:\folder2\ but the missing file test.txt is not showing in the results :
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

$master = Get-ChildItem $PrimaryPath -Recurse  -force
$ComparePath | % { 
   Compare-Object -ReferenceObject $master -DifferenceObject (Get-ChildItem $_ -Recurse | ? { !$_.PSIsContainer }) -Property Name, Length, LastWriteTime -PassThru | where { $_.SideIndicator -eq "=>" } | select fullname 
}

Open in new window

0
 
LVL 40

Expert Comment

by:footech
ID: 38812085
If you change the sideindicator to "<=" it should show files that exist in the master that don't show in $comparepath.
0
 

Author Comment

by:greetrufus
ID: 38812373
Tried that and get very strange results.
Plus, i need the output to be the folder\file that is missing.

Example output:

Comparing c:\folder1 to c:\folder2
 MISSING FILE:  C:\folder2\test.txt

Comparing c:\folder1 to c:\folder3
 MISSING FILE:  C:\folder3\test.txt
0
 
LVL 40

Expert Comment

by:footech
ID: 38812699
What results are you seeing?  In my tests I didn't see anything unexpected.  I'll try running your code to see if it points out anything that I'm not thinking of.

Unfortunately Compare-Object won't show objects in that way (it shows objects that are present, not ones that aren't - you just have to infer which ones are missing).  I would say try to find a way to use the output as I pointed out, or we'll need to hack up the output to try to get it into something you can use (for instance - modifying the string that is returned by the fullname property, I think by using a split and replace).
0
 

Author Comment

by:greetrufus
ID: 38817178
Also, another solution I found that works too
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

#Get filelist with relativepath property
function Get-FilesWithRelativePath ($Path) {
    Get-ChildItem $Path -Recurse | ? { !$_.PSIsContainer } | % { 
        Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.Substring($Path.Length)
        $_
    }
}

#If path exists and is folder
if (Test-Path $PrimaryPath -PathType Container) {
    #Get master fileslist
    $Masterfiles = Get-FilesWithRelativePath (Resolve-Path $PrimaryPath).Path

    #Compare folders
    foreach ($Folder in $ComparePath) {
        if (Test-Path $Folder -PathType Container) {
            #Getting filelist and adding relative-path property to files
            $ResolvedFolder = (Resolve-Path $Folder).Path
            $Files = Get-FilesWithRelativePath $ResolvedFolder

            #Compare and output filepath to missing or old file
            Compare-Object -ReferenceObject $Masterfiles -DifferenceObject $Files -Property RelativePath, Length, LastWriteTime | ? { $_.SideIndicator -eq "<=" } | Select @{n="FilePath";e={Join-Path $ResolvedFolder $_.RelativePath}}
        } else { Write-Error "$Folder is not a valid foldername. Foldertype: Compare" }
    }

} else { Write-Error "$PrimaryPath is not a valid foldername. Foldertype: Master" }

Open in new window

0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will help you understand what HashTables are and how to use them in PowerShell.
Windows 10 came with  a lot of built in applications, Some organisations leave them there, some will control them using GPO's. This Article is useful for those who do not want to have any applications in their image (example:me).
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…
This is a high-level webinar that covers the history of enterprise open source database use. It addresses both the advantages companies see in using open source database technologies, as well as the fears and reservations they might have. In this…

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question