Solved

Remove double lines in a file (grep, sed ...)

Posted on 2001-07-19
3
251 Views
Last Modified: 2007-12-19
Using the grep command, how can I do to suppress the double lines in a input file.
Thanks for your answers.
0
Comment
Question by:kindelberger
3 Comments
 
LVL 51

Expert Comment

by:ahoffmann
ID: 6297777
uniq inputfile
# or if it the double lines are not following each other
sort inputfile|uniq
0
 

Author Comment

by:kindelberger
ID: 6298100
Thanks for your help, but the lines are not following each other and I can not use the sort command because I need to keep the existing order of the lines.
Is there another solution for my  problem ?
Thanks.
0
 
LVL 3

Accepted Solution

by:
gcauthon earned 25 total points
ID: 6298168
use the nl command too like so:

cat testfile |nl |sort +1 |uniq -s 6 |sort +0n |sed 's/^[    0-9]*//'

by the way, the pattern for sed contains a space and a tab to remove the number introduced by nl.  The line numbers basically keep the original ordering so that the lines can be sorted and uniq'd and then rearranged back the way they were.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded‚Ķ
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question