Solved

Remove double lines in a file (grep, sed ...)

Posted on 2001-07-19
3
244 Views
Last Modified: 2007-12-19
Using the grep command, how can I do to suppress the double lines in a input file.
Thanks for your answers.
0
Comment
Question by:kindelberger
3 Comments
 
LVL 51

Expert Comment

by:ahoffmann
Comment Utility
uniq inputfile
# or if it the double lines are not following each other
sort inputfile|uniq
0
 

Author Comment

by:kindelberger
Comment Utility
Thanks for your help, but the lines are not following each other and I can not use the sort command because I need to keep the existing order of the lines.
Is there another solution for my  problem ?
Thanks.
0
 
LVL 3

Accepted Solution

by:
gcauthon earned 25 total points
Comment Utility
use the nl command too like so:

cat testfile |nl |sort +1 |uniq -s 6 |sort +0n |sed 's/^[    0-9]*//'

by the way, the pattern for sed contains a space and a tab to remove the number introduced by nl.  The line numbers basically keep the original ordering so that the lines can be sorted and uniq'd and then rearranged back the way they were.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
This tech tip describes how to install the Solaris Operating System from a tape backup that was created using the Solaris flash archive utility. I have used this procedure on the Solaris 8 and 9 OS, and it shoudl also work well on the Solaris 10 rel…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now