Avatar of Valleriani
Valleriani
Flag for Sweden asked on

Linux Sort/Dedupe but with a header on line 1

I have the following command:

sort FILENAME -u >> FILENAME-2


What it is meant to do is DEDUP AND SORT. However, my files have a one line header at the top; EX.

Email,Fname,Lname
rvalleriani@test.com,Chris,Valler
test@test.com,Mike,Liuk
m@m.com,Maurico,Hammer


The above command will randomize everything including the first line, which it shouldn't do!!

I was told to try this:

tail +2 FILENAME| sort FILENAME -u >> FILENAME-2

However, this doesn't seem to work either.

Any suggestions?
LinuxLinux OS DevLinux Networking

Avatar of undefined
Last Comment
Valleriani

8/22/2022 - Mon
farzanj

try this

sed -n '2,$p' FILENAME | sort -u > FILENAME2
Valleriani

ASKER
Hello!

Instead of keeping it, it actually removes the first line so

email
aaa
zzz
bbb

becomes

aaa
bbb
zzz

with no header.
Valleriani

ASKER
I've requested that this question be deleted for the following reason:

I have gotten the answer, sorry about that, here it is though:<br /><br />gawk 'NR==1; {if(NR &gt; 1) {print $0 | &quot;sort -u&quot;}}' f_edit.txt &gt;&gt; f_edit2.txt
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
ASKER CERTIFIED SOLUTION
farzanj

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
farzanj

Would the last one work for you?
farzanj

When you ask a question, you have to be patient.  People spend time in providing you a solution and you simply waste their time.
Valleriani

ASKER
echo -e $(head -1 FILE)"\n$(sed -n '2,$p' FILE | sort -u)" works as well by farzanj, actually , I had submitted an ticket but apparently, I didn't realize there is an object button.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.