Link to home
Start Free TrialLog in
Avatar of jl66
jl66Flag for United States of America

asked on

how to sort and remove duplicates with Perl

Have a list like the following:
XY-B
AB_C
ABB
ACB
ABB
XYZ
would like to get
ABB
AB_C
ACB
XY-B
XYZ
Prefer to a one-liner.
Avatar of jl66
jl66
Flag of United States of America image

ASKER

The list may contain numbers. Sorry for that.
SOLUTION
Avatar of Adam314
Adam314

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial

perl -ne '$a=$_;s/[^a-zA-Z0-9]//g;$h{$_}=$a;END{print map {$h{$_}} sort keys %h}'

Open in new window

SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of ozo
perl -lne '$s{$_}++;END{print for sort keys %s}' list
Avatar of jl66

ASKER

bouns, the basic sort is good enough.
ozo, if the list is a file, is the one-liner for a file different from the above? I tried to use it for a file, it gave the unexpected result.
in what way was the result unexpected?
Avatar of Adam314
Adam314

The sort that bounsy gives will not remove duplicates.

If you are on unix, you could use uniq and sort commands.
Avatar of jl66

ASKER

ozo,  if I used yours exactly, I got the error:
perl -lne '$s{$_}++;END{print for sort keys %s}' test.txt
Can't find string terminator "'" anywhere before EOF at -e line 1.
if I replaced ' with ", I got the error:
perl -lne "$s{$_}++;END{print for sort keys %s}"   test.txt
Substitution pattern not terminated at -e line 1.
Avatar of jl66

ASKER

Adam314, unfortunately the OS is Windows.
Avatar of jl66

ASKER

bounsy's solution sorted the inputs but kept the duplicates as Adam314 mentioned.
On windows, the command ozo gave would be this.  Replace test.txt with the actual input file name.  If you want the output to go to a file instead, the the second command, replacing output.txt with the desired output file name.

It looks like you tried this already.  Did you try it exactly as it is there?  Or could there have been a typo?  Try using copy/paste to make sure there are no typos.


#output to screen:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt
 
#output to file:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt > output.txt

Open in new window

ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial