Remove duplicate lines from text file using vbscript?

Remove Duplicate Lines From Text File.

How can i remove duplicate lines from text file?
The format of the file is as follows

I am using VBS Script
Who is Participating?
sirbountyConnect With a Mentor Commented:
Try this...

Dim objFSO: Set objFSO = CreateObject("Scripting.FileSystemObject")
Dim objFile: Set objFile = objFSO.OpenTextFile(strFile)
Dim dicSort: Set dicSort = CreateObject("Scripting.Dictionary")

Do While Not objFile.AtEndOfStream
    On Error Resume Next
    strData = objFile.ReadLine
    dicSort.Add strData, dicSort.Count

Set objFile = objFSO.CreateTextFile(strFile)
For Each Item In dicSort
  objFile.WriteLine Item
Set objFile = Nothing
Set objFSO=Nothing
samopalConnect With a Mentor Commented:
  set cnCSV = CreateObject("ADODB.Connection")
   cnCSV.ConnectionString = "Driver={Microsoft Text Driver (*.txt; *.csv)};Initial Catalog=."
   cnCSV.Execute "SELECT distinct * INTO OUT.txt FROM IN.txt"
   Set cnCSV = Nothing
Hi guys, hi Sir Bounty.
Sir Bounty, Im trying to following what you have done. Is it possible to clarify for me the following?

1) You create a dictionary object
2) You then read into the dictionary each line of the text file
What im not getting is, are you writing in every duplicate line into the dictionary object?

Any help clarifying much appreciated mate.
Well Simon - this is quite old! :^)
But reading over the code above, it will error with duplicates - a dictionary won't allow duplicate indexes - which is why the 'on error' line (so any duplicates will simply be skipped).
Hi Sirbounty,
As usual, you're everywhere and always in a good way.
Thanks for clearing that up.
Take care :>)
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.