looking for a script that will compare 2 text files and remove duplicates

Posted on 2010-03-28
Medium Priority
Last Modified: 2012-05-09
I am looking for a scipt that will take a list of servers (list1.txt) and compare it to a second list (list2.txt) if a server is in list 1 I want it deleted from the output so basicily if something is in BOTH lists then i want it not in the out put
Question by:NAMEWITHELD12
  • 2
  • 2
LVL 93

Expert Comment

by:Patrick Matthews
ID: 28921864
Do you want:

1) In List1 but not List2
2) In List2 but not List1
3) (In List1 but not List2) And (In List2 but not List1)


Author Comment

ID: 28923545
1) no
2) yes
3) none of that

and if it is in both list i dont want it
thanks I appreciate the help , I am learning this kinda thing right now and could use the help!!!
LVL 93

Accepted Solution

Patrick Matthews earned 1200 total points
ID: 28924191
OK, this is VBScript.

Dim fso, tsIn, tsOut, dic, entry

Set fso = CreateObject("Scripting.FileSystemObject")
Set dic = CreateObject("Scripting.Dictionary")
dic.CompareMode = 1

Set tsIn = fso.OpenTextFile("c:\folder\subfolder\List1.txt")

Do Until tsIn.AtEndOfStream
    entry = tsIn.ReadLine
    If Not dic.Exists(entry) Then dic.Add entry, entry

Set tsIn = fso.OpenTextFile("c:\folder\subfolder\List1.txt")
Set tsOut = fso.CreateTextFile("c:\folder\subfolder\Results.txt", True)

Do Until tsIn.AtEndOfStream
    entry = tsIn.ReadLine
    If Not dic.Exists(entry) Then tsOut.WriteLine entry

Set tsIn = Nothing
Set tsOut = Nothing
Set fso = Nothing
Set dic = Nothing

MsgBox "Done"

Open in new window

LVL 16

Assisted Solution

t0t0 earned 800 total points
ID: 28926807
Herels a batch file version

@echo off

del "list.tmp" 2>nul

for /f "tokens=* usebackq" %%a in ("list1.txt") do (
   findstr "\<%%a\>" "list2.txt" >nul
   if errorlevel 1 echo %%a>>"list.tmp"

move "list.tmp" "list1.txt"

NOTE: Depending on the contents of your lists, you might be able to get away with using just "%%a" instead of "\<%%a\>" in which case, you can also use FIND instead of FINDSTR.

NOTE: Also, if there are no spaces in your filenames then you could use:

@echo off

del list.tmp 2>nul

for /f "tokens=*" %%a in (list1.txt) do (
   find "%%a" list2.txt >nul
   if errorlevel 1 echo %%a>>list.tmp

move list.tmp list1.txt

NOTE: I also changed FINDSTR to FIND etc,,

Author Closing Comment

ID: 31708142
AWESOME thanks , i am looking at each part of the script to understand what each line does , thanks again

Featured Post

Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
Today, the web development industry is booming, and many people consider it to be their vocation. The question you may be asking yourself is – how do I become a web developer?
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to go…
Starting up a Project

601 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question