Solved

Compare multiple folders for file differences

Posted on 2013-01-21
11
942 Views
Last Modified: 2013-01-24
I began to compare 2 folder structures to find files that did not match by date and size, but the requirment has been changed to 4 folders and I am stuck.

So here is what I am trying to do:
We upload several hundred folders\files to 4 different servers.  The files must all match.  Sometimes a file will not copy properly.  So I need a script to read all four directories and compare all the files to make sure they match by size and date.
Output should only be a simple list that shows me the files that didn't match.

Any ideas?
Thanks.

I can do two folders but am lost on four.  Also, this output is confusing.  Not sure how to only list those that don't match.
$path1 = "\\path\folder
$path2 = "\\path\folder1
$dif = Compare-Object -ReferenceObject $path1 -DifferenceObject $path2 -Property FullName, Length, LastWriteTime
$dif | ft -AutoSize

Open in new window

0
Comment
Question by:greetrufus
  • 5
  • 4
  • 2
11 Comments
 
LVL 39

Expert Comment

by:footech
ID: 38803839
Pretty much the only way I think this can be done is to run the compare one pair at a time.
The output shows everything that is different, so it is showing only those that don't match.  If there is a file called test.txt that is 1 byte in folder A but is 2 bytes in folder B, it will tell you about the file in each folder.  To make this a little more useful in your case, if folder A is the standard, then we want to see anything in folder A for which there isn't a exact match in folder B.  We can do this by filtering the results and looking at the SideIndicator, which shows whether the difference is from the ReferenceObject (left side) or DifferenceObject (right side).
If we need the full path of the file, we'll need to pass the object and look at it's properties to retrieve it.

$source = "\\sourceserver\folder"
$destinations = "\\server1\folder1", "\\server2\folder2", "\\server3\folder3", "\\server4\folder4"
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

0
 
LVL 40

Expert Comment

by:Subsun
ID: 38804558
If the files are copied from One share to other 3 shares and you are trying to find if any files are missing then footech's logic will work. Else if the folders are updated individually then you may have to compare the other folders too. I mean, compare 2nd  folder with 3rd and 4th and 3rd with 4th for difference.
0
 

Author Comment

by:greetrufus
ID: 38806027
I see the problem.  So if is use one as master and compare the others to master, that should work.  I am trying to read the paths in as parameters but this doesnt' seem to be working.

test.pl -p C:\testing\folder1 -c C:\testing\folder2, C:\testing\folder3

Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=") | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
} 
}

Open in new window

0
 
LVL 40

Expert Comment

by:Subsun
ID: 38806060
Try this and see.. test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
0
 

Author Comment

by:greetrufus
ID: 38806080
Runs but empty results.
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 39

Accepted Solution

by:
footech earned 500 total points
ID: 38807031
When I posted my code I hadn't closed the scriptblock for the Where cmdlet.  Looks like you may have added a brace at the end.
Here's the code that works.
Param(
    [parameter(Mandatory=$true)]
    [alias("p")]
    $PrimaryPath,
    [parameter(Mandatory=$true)]
    [alias("c")]
    $ComparePath)
	
$source = $PrimaryPath
$destinations = $ComparePath
ForEach ($dest in $destinations)
{
  $dif = Compare-Object -ReferenceObject (gci $source -recurse) -DifferenceObject (gci $dest -recurse) -Property FullName, Length, LastWriteTime -passthru | Where {($_.SideIndicator -eq "<=")} | ForEach { $_.fullname }
  Write-Output "Files missing from or different under $dest"
  Write-Output $dif
}

Open in new window

I had assumed that you were already generating the objects to be compared and worked out the compare syntax, so I didn't test or put anything special in for that.  Just to point out that if you're running
test.ps1 -p C:\testing\folder1 -c "C:\testing\folder2","C:\testing\folder3"
all the files should show since the fullname with the path will always be different.  I wonder if we're going to have to work out something different where we compare the source and destination folder by folder, just looking at the name property instead of the fullname.
0
 

Author Comment

by:greetrufus
ID: 38811919
I came up with this, that will show files that don't match, but it is not showing files that don't exist.
For example: c:\folder1\test.txt does not exist on c:\folder2\ but the missing file test.txt is not showing in the results :
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

$master = Get-ChildItem $PrimaryPath -Recurse  -force
$ComparePath | % { 
   Compare-Object -ReferenceObject $master -DifferenceObject (Get-ChildItem $_ -Recurse | ? { !$_.PSIsContainer }) -Property Name, Length, LastWriteTime -PassThru | where { $_.SideIndicator -eq "=>" } | select fullname 
}

Open in new window

0
 
LVL 39

Expert Comment

by:footech
ID: 38812085
If you change the sideindicator to "<=" it should show files that exist in the master that don't show in $comparepath.
0
 

Author Comment

by:greetrufus
ID: 38812373
Tried that and get very strange results.
Plus, i need the output to be the folder\file that is missing.

Example output:

Comparing c:\folder1 to c:\folder2
 MISSING FILE:  C:\folder2\test.txt

Comparing c:\folder1 to c:\folder3
 MISSING FILE:  C:\folder3\test.txt
0
 
LVL 39

Expert Comment

by:footech
ID: 38812699
What results are you seeing?  In my tests I didn't see anything unexpected.  I'll try running your code to see if it points out anything that I'm not thinking of.

Unfortunately Compare-Object won't show objects in that way (it shows objects that are present, not ones that aren't - you just have to infer which ones are missing).  I would say try to find a way to use the output as I pointed out, or we'll need to hack up the output to try to get it into something you can use (for instance - modifying the string that is returned by the fullname property, I think by using a split and replace).
0
 

Author Comment

by:greetrufus
ID: 38817178
Also, another solution I found that works too
Param( 
    [parameter(Mandatory=$true)] [alias("p")] [string]$PrimaryPath,
    [parameter(Mandatory=$true)] [alias("c")] [string[]]$ComparePath
    ) 

#Get filelist with relativepath property
function Get-FilesWithRelativePath ($Path) {
    Get-ChildItem $Path -Recurse | ? { !$_.PSIsContainer } | % { 
        Add-Member -InputObject $_ -MemberType NoteProperty -Name RelativePath -Value $_.FullName.Substring($Path.Length)
        $_
    }
}

#If path exists and is folder
if (Test-Path $PrimaryPath -PathType Container) {
    #Get master fileslist
    $Masterfiles = Get-FilesWithRelativePath (Resolve-Path $PrimaryPath).Path

    #Compare folders
    foreach ($Folder in $ComparePath) {
        if (Test-Path $Folder -PathType Container) {
            #Getting filelist and adding relative-path property to files
            $ResolvedFolder = (Resolve-Path $Folder).Path
            $Files = Get-FilesWithRelativePath $ResolvedFolder

            #Compare and output filepath to missing or old file
            Compare-Object -ReferenceObject $Masterfiles -DifferenceObject $Files -Property RelativePath, Length, LastWriteTime | ? { $_.SideIndicator -eq "<=" } | Select @{n="FilePath";e={Join-Path $ResolvedFolder $_.RelativePath}}
        } else { Write-Error "$Folder is not a valid foldername. Foldertype: Compare" }
    }

} else { Write-Error "$PrimaryPath is not a valid foldername. Foldertype: Master" }

Open in new window

0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

This is a PowerShell web interface I use to manage some task as a network administrator. Clicking an action button on the left frame will display a form in the middle frame to input some data in textboxes, process this data in PowerShell and display…
I thought I'd write this up for anyone who has a request to create an anonymous whistle-blower-type submission form created using SharePoint 2010 (this would probably work the same for 2013). It's not 100% fool-proof but it's as close as you can get…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now