Solved

Compare CSV files using Powershell

Posted on 2013-11-13
4
483 Views
Last Modified: 2013-11-13
I have two very large CSV files both with the following headings 'Name', FullName','Length'.
What I am looking for is a way to compare these CSV files in Powershell.

I need to know...

1) What files are unique to each set (Based purely on Name).
2) What files exist in both but whos lengths are different (Based on Name and Length).

Basically a report on what changes are represented between the two CSV files... If the results could be output to seperate files representing each type of difference that would be ideal?

This is to solve a problem where a new version of software has grown rapidly in size but the cause of the growth is unknown.
0
Comment
Question by:Blowfelt82
  • 2
  • 2
4 Comments
 
LVL 39

Expert Comment

by:footech
ID: 39644221
Let me know how the below works for you.  The output files only include the file names.  If you need it to be something else, let me know if you need help making the adjustment to the code.  Hope that all files names are unique in each .CSV, otherwise this won't work.
$file1 = Import-CSV file1.csv
$file2 = Import-CSV file2.csv
Compare-Object $file1 $file2 -Property Name | Select -ExpandProperty Name | Out-File UniqueFiles.txt
($file1 + $file2) | Group -Property Name |
 ? { $_.count -eq 2 } |
 % { Compare-Object ($_.group)[0] ($_.group)[1] -property Name,Length -passthru } |
 Select -ExpandProperty Name -ExcludeProperty SideIndicator -Unique |
 Out-File ChangedSize.txt

Open in new window

0
 

Author Comment

by:Blowfelt82
ID: 39644296
Its basically a comparison of a c:\ drive exported from a wim file, so there may well be duplicated names... Perhaps using the fullpath field would give greater acuracy?
0
 
LVL 39

Accepted Solution

by:
footech earned 500 total points
ID: 39645338
Using the fullname would avoid errors.  All you would have to do to modify the script is change each instance of "Name" to "FullName".
0
 

Author Closing Comment

by:Blowfelt82
ID: 39645462
Thanks again for your help
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I thought I'd write this up for anyone who has a request to create an anonymous whistle-blower-type submission form created using SharePoint 2010 (this would probably work the same for 2013). It's not 100% fool-proof but it's as close as you can get…
This script can help you clean up your user profile database by comparing profiles to Active Directory users in a particular OU, and removing the profiles that don't match.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question