We help IT Professionals succeed at work.

Wierd characters in a file.

sunshine737
sunshine737 used Ask the Experts™
on
am looking for help in removing the weird characters
which are coming along with special characters in the source file .

The user enters text for a description field manually
the description may consist special characters like registered mark( R) , trademark ( TM)
and even some times for regular pipe symbol( | )

the characters are coming good, but a non printable weird character is coming along with
these special characters.
Please verify the image in attachment
you can see a wierd character looks like 'A' between Staples and Registerd mark
and also the character like 'A' between Post-it and Registerd marks

I want to remove such kind of wierd characters only, before processing my source file

I appreciate your help to remove these wierd characters preferably in unix

unix environment: IBM AIX

Thanks wierd characters in a file
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Commented:
tr -cd '\11\12\40-\176' < $INPUT_FILE > $OUTPUT_FILE

Commented:
you can also use the [:printable:] filter in the tr instead of the \xx\yyy\....

Author

Commented:
Sorry,that didn't worked.
I am attaching the sample file.i can see special characters,when i open the file in unix.

Thanks
wierd-characters-file.txt
Commented:
Hello,

Try this.

sed 's/Â//g' abc > file1 --> It will deletes the character  and send the output to file1
mv -f file1 abc --> Renaming the new file to the original file name.

I believe you have to do this manually for all the special characters in the file or else place all the special characters in a file and can create a script to run ans check all the special characters.

Let me know if this works.

Thanks,
MSK.