Substitutions involving non-ASCII characters

I have a file with some binary characters that are unrecognizable in any of my text editors.

For example, I would see something like this:

\xC2Freien - with \xC2 being some binary character. I attempted to make this substitution in my processing script to clean these out:

s/[^[:ascii:]]Freien/Freien/g;

However, it failed to do so. I know I can do something like this:

s/[^[:ascii:]]//g;

While this will get rid of those binary characters, it will also strip out non-ASCII characters I do what to keep.

I know I can also do hexadecimal substitution like this:

s/\xC2Freien/Freien/g;

However, in future cases I won't always know the hexadecimal value.

I can't cut & paste binary characters like this because \xC2 will transform into this: ? in the cut & paste process
hadronsAsked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
 
ddrudikConnect With a Mentor Commented:
Determine your range of disallowed characters (refer to ASCII chart for your desired range) and specify as such:
s/[\xC2-\xCF]//g;

Open in new window


You could also specify the negation of your allowed range if it's quicker to specify that (replace with your desired range):
s/[^\x00-\x7F]//g;

Open in new window

0
 
hadronsAuthor Commented:
s/[^\x00-\x7F]Freien/Freien/g; is the winner; I didn't have any results with the other expression ... and I know of the [^\x00-\x7F], but I didn't think to use it. Thanks
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.