?
Solved

Compare multiple folders for file differences

Posted on 2013-01-21
11
Medium Priority
?
1,174 Views
Last Modified: 2013-01-24
I began to compare 2 folder structures to find files that did not match by date and size, but the requirment has been changed to 4 folders and I am stuck.

So here is what I am trying to do:
We upload several hundred folders\files to 4 different servers.  The files must all match.  Sometimes a file will not copy properly.  So I need a script to read all four directories and compare all the files to make sure they match by size and date.
Output should only be a simple list that shows me the files that didn't match.

Any ideas?
Thanks.

I can do two folders but am lost on four.  Also, this output is confusing.  Not sure how to only list those that don't match.
$path1 = "\\path\folder
$path2 = "\\path\folder1
$dif = Compare-Object -ReferenceObject $path1 -DifferenceObject $path2 -Property FullName, Length, LastWriteTime
$dif | ft -AutoSize

Open in new window

0
Comment
Question by:greetrufus
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 2
11 Comments
 
LVL 41

Expert Comment

by:footech
ID: 38803839
Pretty much the only way I think this can be done is to run the compare one pair at a time.
The output shows everything that is different, so it is showing only those that don't match.  If there is a file called test.txt that is 1 byte in folder A but is 2 bytes in folder B, it will tell you about the file in each folder.  To make this a little more useful in your case, if folder A is the standard, then we want to see anything in folder A for which there isn't a exact match in folder B.  We can do this by filtering the results and looking at the SideIndicator, which shows whether the difference is from the ReferenceObject (left side) or DifferenceObject (right side).
If we need the full path of the file, we'll need to pass the object and look at it's properties to retrieve it.

$source = "\\sourceserver\folder"
$destinations = "\\server1\folder1", "\\server2\folder2", "\\server3\folder3", "\\server4\folder4"
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

0
 
LVL 40

Expert Comment

by:Subsun
ID: 38804558
If the files are copied from One share to other 3 shares and you are trying to find if any files are missing then footech's logic will work. Else if the folders are updated individually then you may have to compare the other folders too. I mean, compare 2nd  folder with 3rd and 4th and 3rd with 4th for difference.
0
 

Author Comment

by:greetrufus
ID: 38806027
I see the problem.  So if is use one as master and compare the others to master, that should work.  I am trying to read the paths in as parameters but this doesnt' seem to be working.

test.pl -p C:\testing\folder1 -c C:\testing\folder2, C:\testing\folder3

Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
} 
}

Open in new window

0
Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

 
LVL 40

Expert Comment

by:Subsun
ID: 38806060
Try this and see.. test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
0
 

Author Comment

by:greetrufus
ID: 38806080
Runs but empty results.
0
 
LVL 41

Accepted Solution

by:
footech earned 2000 total points
ID: 38807031
When I posted my code I hadn't closed the scriptblock for the Where cmdlet.  Looks like you may have added a brace at the end.
Here's the code that works.
Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=")} | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

I had assumed that you were already generating the objects to be compared and worked out the compare syntax, so I didn't test or put anything special in for that.  Just to point out that if you're running
test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
all the files should show since the fullname with the path will always be different.  I wonder if we're going to have to work out something different where we compare the source and destination folder by folder, just looking at the name property instead of the fullname.
0
 

Author Comment

by:greetrufus
ID: 38811919
I came up with this, that will show files that don't match, but it is not showing files that don't exist.
For example: c:\folder1\test.txt does not exist on c:\folder2\ but the missing file test.txt is not showing in the results :
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

$master = Get-ChildItem $PrimaryPath -Recurse  -force
$ComparePath | % { 
   Compare-Object -ReferenceObject $master -DifferenceObject (Get-ChildItem $_ -Recurse | ? { !$_.PSIsContainer }) -Property Name, Length, LastWriteTime -PassThru | where { $_.SideIndicator -eq "=>" } | select fullname 
}

Open in new window

0
 
LVL 41

Expert Comment

by:footech
ID: 38812085
If you change the sideindicator to "<=" it should show files that exist in the master that don't show in $comparepath.
0
 

Author Comment

by:greetrufus
ID: 38812373
Tried that and get very strange results.
Plus, i need the output to be the folder\file that is missing.

Example output:

Comparing c:\folder1 to c:\folder2
 MISSING FILE:  C:\folder2\test.txt

Comparing c:\folder1 to c:\folder3
 MISSING FILE:  C:\folder3\test.txt
0
 
LVL 41

Expert Comment

by:footech
ID: 38812699
What results are you seeing?  In my tests I didn't see anything unexpected.  I'll try running your code to see if it points out anything that I'm not thinking of.

Unfortunately Compare-Object won't show objects in that way (it shows objects that are present, not ones that aren't - you just have to infer which ones are missing).  I would say try to find a way to use the output as I pointed out, or we'll need to hack up the output to try to get it into something you can use (for instance - modifying the string that is returned by the fullname property, I think by using a split and replace).
0
 

Author Comment

by:greetrufus
ID: 38817178
Also, another solution I found that works too
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

#Get filelist with relativepath property
function Get-FilesWithRelativePath ($Path) {
    Get-ChildItem $Path -Recurse | ? { !$_.PSIsContainer } | % { 
        Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.Substring($Path.Length)
        $_
    }
}

#If path exists and is folder
if (Test-Path $PrimaryPath -PathType Container) {
    #Get master fileslist
    $Masterfiles = Get-FilesWithRelativePath (Resolve-Path $PrimaryPath).Path

    #Compare folders
    foreach ($Folder in $ComparePath) {
        if (Test-Path $Folder -PathType Container) {
            #Getting filelist and adding relative-path property to files
            $ResolvedFolder = (Resolve-Path $Folder).Path
            $Files = Get-FilesWithRelativePath $ResolvedFolder

            #Compare and output filepath to missing or old file
            Compare-Object -ReferenceObject $Masterfiles -DifferenceObject $Files -Property RelativePath, Length, LastWriteTime | ? { $_.SideIndicator -eq "<=" } | Select @{n="FilePath";e={Join-Path $ResolvedFolder $_.RelativePath}}
        } else { Write-Error "$Folder is not a valid foldername. Foldertype: Compare" }
    }

} else { Write-Error "$PrimaryPath is not a valid foldername. Foldertype: Master" }

Open in new window

0

Featured Post

Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A procedure for exporting installed hotfix details of remote computers using powershell
A recent project that involved parsing Tableau Desktop and Server log files to extract reusable user queries for use in other systems. I chose to use PowerShell to gather the data, and SharePoint to present it...
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…
Want to learn how to record your desktop screen without having to use an outside camera. Click on this video and learn how to use the cool google extension called "Screencastify"! Step 1: Open a new google tab Step 2: Go to the left hand upper corn…

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question