We help IT Professionals succeed at work.

regex to remove strange character from text file

omcr
omcr asked
on
Medium Priority
507 Views
Last Modified: 2012-06-21
Each line of text in log file I get from a device has a small rectangle at the begining, how do I get rid of it? Here is what it looks like, just imagine the zero's are small rectangles:

0SeqNum: 57
0Display: 1
0Point: 9
0CurrentState: Alarm

I have chomped, choped, tried to strip out with regexs, and have had no success.
Comment
Watch Question

ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015

Commented:
what do you see if you do
while( <> ){
  s/([^\n -z])/sprintf'\x%02x',ord $1/eg;
  print;
}

Author

Commented:
That gave me this:
\x0dSeqNum: 81\x0d
\x0dDisplay: 1\x0d
\x0dPoint: 9\x0d
\x0dCurrentState: Alarm\x0d
ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015

Commented:
"\0xd" is a carriage return, was the log file generated in DOS?
perl -i.bak -pe 's/\xd//g' logfile

Author

Commented:
generated in unix.
Can you put "perl -i.bak -pe 's/\xd//g' logfile" into a file format instead of command line?
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015
Commented:
#!/usr/bin/perl
{local @ARGV=<*.log>;
 local $I^='';
 while( <> ){
    s/\xd//g;
    print;
 }
}

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts
Kim RyanIT Consultant

Commented:
Can you dump the file so we can identify the strange character?
od -s logfile
od -x logfile

If you are on unix, thr tr utility may be simpler. The first example removes all control characters. The second, all characters with octal value of 015

cat logfile | tr -d '[:control:]' > new_logfile
cat logfile | tr -d '\015' > new_logfile

Author

Commented:
That did it. Thanks.

Author

Commented:
Did not see your post before I commented last time teraplane, but ozo's suggestion cleaned it up, and my scripts are now working. Thanks.
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.