Solved

Remove double lines in a file (grep, sed ...)

Posted on 2001-07-19
3
245 Views
Last Modified: 2007-12-19
Using the grep command, how can I do to suppress the double lines in a input file.
Thanks for your answers.
0
Comment
Question by:kindelberger
3 Comments
 
LVL 51

Expert Comment

by:ahoffmann
ID: 6297777
uniq inputfile
# or if it the double lines are not following each other
sort inputfile|uniq
0
 

Author Comment

by:kindelberger
ID: 6298100
Thanks for your help, but the lines are not following each other and I can not use the sort command because I need to keep the existing order of the lines.
Is there another solution for my  problem ?
Thanks.
0
 
LVL 3

Accepted Solution

by:
gcauthon earned 25 total points
ID: 6298168
use the nl command too like so:

cat testfile |nl |sort +1 |uniq -s 6 |sort +0n |sed 's/^[    0-9]*//'

by the way, the pattern for sed contains a space and a tab to remove the number introduced by nl.  The line numbers basically keep the original ordering so that the lines can be sorted and uniq'd and then rearranged back the way they were.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Best software for Unix commands in Windows cmd.exe? 18 102
How to find files recursively in ftp server quickly? 7 99
Java core in Solaris 10 1 262
Python Assistance 7 73
Let's say you need to move the data of a file system from one partition to another. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now