jl66
asked on
how to sort and remove duplicates with Perl
Have a list like the following:
XY-B
AB_C
ABB
ACB
ABB
XYZ
would like to get
ABB
AB_C
ACB
XY-B
XYZ
Prefer to a one-liner.
XY-B
AB_C
ABB
ACB
ABB
XYZ
would like to get
ABB
AB_C
ACB
XY-B
XYZ
Prefer to a one-liner.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
perl -ne '$a=$_;s/[^a-zA-Z0-9]//g;$h{$_}=$a;END{print map {$h{$_}} sort keys %h}'
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
perl -lne '$s{$_}++;END{print for sort keys %s}' list
ASKER
bouns, the basic sort is good enough.
ozo, if the list is a file, is the one-liner for a file different from the above? I tried to use it for a file, it gave the unexpected result.
ozo, if the list is a file, is the one-liner for a file different from the above? I tried to use it for a file, it gave the unexpected result.
in what way was the result unexpected?
The sort that bounsy gives will not remove duplicates.
If you are on unix, you could use uniq and sort commands.
If you are on unix, you could use uniq and sort commands.
ASKER
ozo, if I used yours exactly, I got the error:
perl -lne '$s{$_}++;END{print for sort keys %s}' test.txt
Can't find string terminator "'" anywhere before EOF at -e line 1.
if I replaced ' with ", I got the error:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt
Substitution pattern not terminated at -e line 1.
perl -lne '$s{$_}++;END{print for sort keys %s}' test.txt
Can't find string terminator "'" anywhere before EOF at -e line 1.
if I replaced ' with ", I got the error:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt
Substitution pattern not terminated at -e line 1.
ASKER
Adam314, unfortunately the OS is Windows.
ASKER
bounsy's solution sorted the inputs but kept the duplicates as Adam314 mentioned.
On windows, the command ozo gave would be this. Replace test.txt with the actual input file name. If you want the output to go to a file instead, the the second command, replacing output.txt with the desired output file name.
It looks like you tried this already. Did you try it exactly as it is there? Or could there have been a typo? Try using copy/paste to make sure there are no typos.
It looks like you tried this already. Did you try it exactly as it is there? Or could there have been a typo? Try using copy/paste to make sure there are no typos.
#output to screen:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt
#output to file:
perl -lne "$s{$_}++;END{print for sort keys %s}" test.txt > output.txt
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER