?
Solved

How to remove ^M characters from a file

Posted on 2004-11-24
21
Medium Priority
?
1,956 Views
Last Modified: 2012-08-13
There Control M(^M) characters in a file along with blank spaces
ie. "                                        ^M"
     <----------blank spaces ----->
I need to remove all the lines in the file where there are blank spaces with ^M and also new lines with ^M characters.

Pls help.

prasen
0
Comment
Question by:prasen120998
17 Comments
 

Author Comment

by:prasen120998
ID: 12666810
The OS is AIX 4.3, its a .txt file
0
 
LVL 62

Expert Comment

by:gheist
ID: 12666848
tr -d \013 < badfile > goodfile
0
 
LVL 19

Expert Comment

by:ramazanyich
ID: 12666962
dos2unix your_in_file > out_file
out_file will be without ^M
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 38

Expert Comment

by:yuzh
ID: 12671083
For AIX, you see the answer in:
you can use:

 sed s/^M// file > newfile
mv newfile file


0
 
LVL 38

Expert Comment

by:yuzh
ID: 12671086
also see
http:Q_21000845.html

to learn more.
0
 
LVL 4

Accepted Solution

by:
bytta earned 252 total points
ID: 12678000
I guess I'm the first one to get rid of the white spaces too...
# removes ^M and throws away all lines not containing any letter or number (probably what you want)
 sed s/^M// file | grep  [a-zA-Z0-9] > newfile

# Peronally I like to do it using vim (no newfile made) - but if I remember correctly this reduces
"                                        ^M" (spaces or tabs) to an empty line, but doesn't remove it
vi -e -c "%s/[ \t]*^M//g" -c "wq" -c 2 file1

Note that ^M isn't a "^" and a "M" but a meta char so you can't copy/paste it - create it in the command line by pressing Ctrl-V and then Enter

0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 12686884
yuzh: a bit lazy ;-)

 sed '/^M/d' file > newfile

# where ^M is a real carriage return, depending on your shell. Most likely  Ctrl-V followed by Ctrl-J works
0
 
LVL 1

Assisted Solution

by:sirjb
sirjb earned 248 total points
ID: 12755291
There is another way to remove all BLANK+^M at once

sed /[[:space:]]*[[:space:]]^M//g file > correctfile


The combination of [[:space:]]*[[:space:]]^M is actually: any combinations of any kind of spaces ( tabs, space, other blank ) end with control+M
This will not replace the control+M char but also all blank spaces in front of it.
remember the control +M => ctrl-V and then Enter.

If you need other combination using sed, let me know.

Hope this can help you.

Regards,

JB
0
 
LVL 62

Expert Comment

by:gheist
ID: 12755751
^m is same \013

0
 
LVL 38

Expert Comment

by:yuzh
ID: 12760054
^M is "\015" !

eg:

tr -d "\015" < infile > outfile
0
 
LVL 1

Expert Comment

by:smiththomasg
ID: 12788863
Not trying to beat a dead horse.  The "tr" method seems best.  Here's a script: "stripCR":

#!/bin/sh
if [ ! $# = 1 ] ;then echo "Usage: stripCR {filename}..." ;exit ;fi
cFile=$1
if [ ! -f $cFile ] ;then echo "No such file..." ;exit ;fi
if tr -d '\r' < $cFile > ./_$$.tmp ;then
  mv ./_$$.tmp $cFile
fi
# End...

^M is CR (Carriage return) so '\r' will work.

0
 

Expert Comment

by:redtwilight04
ID: 12989854
I personally think that dos2unix is still the best command to use to remove the ^M automatically.

Usage: dos2unix inputfile outputfile

Input file and Output file can be the same file or different files.
0
 
LVL 2

Expert Comment

by:danrudolph
ID: 13133770
here are 2 methods

sed -e 's/^M$//' xxx.log > xxx.lognew && mv -f xxx.lognew xxx.log

for x in *; do
mv $x $x.old
sed -e 's/^M$//' $x.old > $x
rm -f $x.old
done

Hope this helps

Dan

0
 
LVL 62

Expert Comment

by:gheist
ID: 13456471
12/06/2004 05:18PM PST
Gives more efficient command line ...
0
 
LVL 20

Expert Comment

by:jmcg
ID: 13456515
Gheist -

Thanks for your comment. The two answers I recommended are the only ones that treated the part of the question about removing lines containing only blanks. As I read the question yet again, it looks like the lines containing only blanks are to be removed entirely. I would question whether either of the two recommended responses actually does what the asker wanted, but it seems pretty clear that those responses offering various ways of stripping out just the \r characters fall short.

jmcg
EE Cleanup Volunteer
0
 
LVL 62

Expert Comment

by:gheist
ID: 13456599
No-one removes empty lines ... ( like grep '^..*' or so (this is wrong..)  )
vim example is same as using sed, aix's vi is able to use it. ( oslevel = 4.3.3-9 )

Question is a bit messy.
> I need to remove all the lines in the file where there are blank spaces with ^M and also new lines with ^M characters
Common sense tells that leaving only lines with one single word is impractical at best, and it all looks like a conversion of PC-DOS file to AIX.
0
 
LVL 62

Expert Comment

by:gheist
ID: 13458465
Maybe if no one cares ...
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses
Course of the Month16 days, 11 hours left to enroll

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question