Solved

Remove double lines in a file (grep, sed ...)

Posted on 2001-07-19
3
250 Views
Last Modified: 2007-12-19
Using the grep command, how can I do to suppress the double lines in a input file.
Thanks for your answers.
0
Comment
Question by:kindelberger
3 Comments
 
LVL 51

Expert Comment

by:ahoffmann
ID: 6297777
uniq inputfile
# or if it the double lines are not following each other
sort inputfile|uniq
0
 

Author Comment

by:kindelberger
ID: 6298100
Thanks for your help, but the lines are not following each other and I can not use the sort command because I need to keep the existing order of the lines.
Is there another solution for my  problem ?
Thanks.
0
 
LVL 3

Accepted Solution

by:
gcauthon earned 25 total points
ID: 6298168
use the nl command too like so:

cat testfile |nl |sort +1 |uniq -s 6 |sort +0n |sed 's/^[    0-9]*//'

by the way, the pattern for sed contains a space and a tab to remove the number introduced by nl.  The line numbers basically keep the original ordering so that the lines can be sorted and uniq'd and then rearranged back the way they were.
0

Featured Post

Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Help stop mail server from sending spam. 27 129
Compare if date string is within text file 3 65
Java core in Solaris 10 1 296
check unix curl command return value 7 151
Hello fellow BSD lovers, I've created a patch process for patching openjdk6 for BSD (FreeBSD specifically), although I tried to keep all BSD versions in mind when creating my patch. Welcome to OpenJDK6 on BSD First let me start with a little …
My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question