Perl code to remove duplicate lines from text file
Posted on 1998-10-10
Ok, I need a quick Perl script that can take a sorted text file and remove duplicate lines and my brain is too fried to write this myself so I'm hoping someone can save me some time.
I have a text file (mailing list) which consists of two fields per line: <email> (space as delimiter) (person's name in parenthesis) - I need a routine to read the text file and remove duplicates in the first field.
I'll also award points if someone can give you a unix solution to this.
This needs to handle a 12,000+ line text file and be case insensitive (in determining dupes).