We help IT Professionals succeed at work.

Check out our new AWS podcast with Certified Expert, Phil Phillips! Listen to "How to Execute a Seamless AWS Migration" on EE or on your favorite podcast platform. Listen Now

x

how to sort and remove duplicates with Perl

jl66
jl66 asked
on
Medium Priority
300 Views
Last Modified: 2012-05-07
Have a list like the following:
XY-B
AB_C
ABB
ACB
ABB
XYZ
would like to get
ABB
AB_C
ACB
XY-B
XYZ
Prefer to a one-liner.
Comment
Watch Question

jl66Consultant

Author

Commented:
The list may contain numbers. Sorry for that.
Top Expert 2009
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

perl -ne '$a=$_;s/[^a-zA-Z0-9]//g;$h{$_}=$a;END{print map {$h{$_}} sort keys %h}'

Open in new window

Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION
ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015

Commented:
perl -lne '$s{$_}++;END{print for sort keys %s}' list
jl66Consultant

Author

Commented:
bouns, the basic sort is good enough.
ozo, if the list is a file, is the one-liner for a file different from the above? I tried to use it for a file, it gave the unexpected result.
ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015

Commented:
in what way was the result unexpected?
Top Expert 2009

Commented:
The sort that bounsy gives will not remove duplicates.

If you are on unix, you could use uniq and sort commands.
jl66Consultant

Author

Commented:
ozo,  if I used yours exactly, I got the error:
perl -lne '$s{$_}++;END{print for sort keys %s}' test.txt
Can't find string terminator "'" anywhere before EOF at -e line 1.
if I replaced ' with ", I got the error:
perl -lne "$s{$_}++;END{print for sort keys %s}"   test.txt
Substitution pattern not terminated at -e line 1.
jl66Consultant

Author

Commented:
Adam314, unfortunately the OS is Windows.
jl66Consultant

Author

Commented:
bounsy's solution sorted the inputs but kept the duplicates as Adam314 mentioned.
Top Expert 2009

Commented:
On windows, the command ozo gave would be this.  Replace test.txt with the actual input file name.  If you want the output to go to a file instead, the the second command, replacing output.txt with the desired output file name.

It looks like you tried this already.  Did you try it exactly as it is there?  Or could there have been a typo?  Try using copy/paste to make sure there are no typos.


#output to screen:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt
 
#output to file:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt > output.txt

Open in new window

CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.