[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 222
  • Last Modified:

One liner---need to remove dups

guess i would have serveral option however what is a good one liner with perl that will remove
duplicate lines in a file.

I have a file with a few thousand lines in it and it has duplicate lines that was created because
of the "case"

Ex

lineone
LineOne
linetwo
LineTwo


how can i change all text to "lower case" then remove and line that is duplicated.

Thanks,

0
bt707
Asked:
bt707
  • 2
1 Solution
 
bt707Author Commented:
I found how to do it using    "tr -d" and "tr -s"  
still curious if there is a simple way to do it with a
perl command.
0
 
manav_mathurCommented:
dunno about a simple Perl command, but

#!/usr/bin/perl
my %hash=() ;
@arr = grep{!$hash{lc $_}++} <> ;
print @arr ;

0
 
bt707Author Commented:
Thanks manav,

sure glad to see your still around.

I found a simple way to do what I needed with
cat testfile777 | tr -s '[:upper:]' '[:lower:]' | sort | uniq > tr-s

was curious how you would do it with perl, as usual there never seem
to be anything you don't have a answer of solution to.

Thanks
0
 
ozoCommented:
perldoc -q "How can I remove duplicate elements from a list or array?"

perl -ne 'print unless $hash{lc}++' testfile777
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now