Automated Script to compare Two Text Files and Save a copy of Only Differences

My dearest Experts,

I want to compare two plain text files.  original.txt and new.txt
original.txt would be a full Customer list from a client a day old, and new.txt will be a full Customer list from a client from today.  I want to generate a script that will look at these two on a daily basis and save a copy of only the data in new.txt that did not exactly exist in original.txt to a file names diff.txt

Example:

original.txt
One
Two
Three
Five
Six
Seven
Eight

Open in new window


new.txt
One
Two
Three
Four
Five
Six
Seven
Eight
Nine
Ten
Eleven

Open in new window


diff.txt
Four
Nine
Ten
Eleven

Open in new window


Is this at all possible?  I see plenty of option on comparing text with other applications, but I want to do this automatically on a scheduled basis every day.  

Also, please keep in mind that my sample is nothing compared to what I'm comparing.  The data files a full customer demographics, and the files are 60,000+ lines of text (comprised of "~" delimited data).

Thank you in advance.

-Nick
NCollinsBBPAsked:
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

x
 
burnocrashConnect With a Mentor Commented:
compare-object -ReferenceObject $(Get-Content .\original.txt) -DifferenceObject $(Get-Content new.txt) | select Inputobject | format-table -Wrap
0
 
ozoCommented:
sort original.txt > oringinal.sort
sort new.txt > new.sort
comm -13 oringinal.sort new.sort > diff.txt
0
 
ozoCommented:
Or, if diff.txt needs to keep the data in the same order as they appeared in new.txt:

perl -ne 'print if !$s{$_}++ && !@ARGV' original.txt new.txt > diff.txt
0
Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

 
burnocrashCommented:
if you wanna do in powershell.

here is the script,

compare-object -ReferenceObject $(Get-Content .\original.txt) -DifferenceObject $(Get-Content new.txt) > diff.txt
0
 
NCollinsBBPAuthor Commented:
@ozo, I do not have the liberty to utilize Perl on my current environment.  

@burnocrash, I have run the following script on my end in PowerShell...

compare-object -ReferenceObject $(Get-Content C:\test\old.txt) -DifferenceObject $(Get-Content new.txt) > C:\test\diff.txt  

Now, I get what I believe is the correct # of lines, but I do not see what I think I should see...

I'm getting in diff.txt a blank line at top, then two headers of "Input Object" as well as "SideIndicator", then my results.  But, I only get the first 56 characters of the line, followed by "...   =>"  

Is it possible to get diff.txt to show JUST the difference results in full?

-Nick
0
 
NCollinsBBPAuthor Commented:
@burnocrash
Success!  (In regards to the output in the PowerShell screen).  Can this be spit out into the "diff.txt" file?  

My reason on doing this is that I receive a full customer file each and every day from a client, which has 60,000+ rows in, where only 75 to 100 of the lines are either updated or brand new.  Importing each of these daily is just killing my processing with the duplicates.  I can save hours in processing if I can just get the differences / new items spit out.  (And the client will not give the resources to change the customer extract... which is why I'm in this boat)

-Nick
0
 
burnocrashConnect With a Mentor Commented:
just add diff.txt to it.

here is the code,

compare-object -ReferenceObject $(Get-Content .\original.txt) -DifferenceObject $(Get-Content new.txt) | select Inputobject | format-table -Wrap > diff.txt

Enjoy :-)
0
 
tel2Commented:
I suggest https:#a39640926 be accepted as the answer, as I see no reason to believe it didn't finish off the job.

Too bad the asker didn't specify the OS in the first place.  Would have saved ozo from wasting his time on it.
0
All Courses

From novice to tech pro — start learning today.