Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions

What is the difference between Western European (ISO) and Unicode (UTF-8) character types?

Posted on 2016-11-18
Last Modified: 2016-11-23
I see that most all character types are set to Western Europe (ISO) (In our On-Prem Exchange and in O365 Exchange Online)  - Is there a reason for this? What implications would occur if we change it to Unicode (UTF-8)?
  • 3
  • 2
  • 2
LVL 93

Expert Comment

by:John Hurst
ID: 41893485
Depends on what character. ISO is single byte (256 characters) and UTF is multi-byte. If you are using only single byte  then you should not see a difference.

Author Comment

by:Health Payment Systems
ID: 41893634
I guess I don't understand when you say single-byte and multi-byte as well as what characters we use? The reason i'm asking is that we have an application that sends out emails. There is working in the subject line which contains the trademark symbol, which for some mail domains, get messed up. It was suggested that possibly changing the character type on our exchange server to Unicode (UTF-8) would fix that. We just don't know if that would also break anything as well.
LVL 93

Assisted Solution

by:John Hurst
John Hurst earned 250 total points
ID: 41893641
A UTF character contains more than 256 bits (512 bits most likely). So long as the first 256 bits are the same for your character sets, you should not have any issue.
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

LVL 16

Accepted Solution

DansDadUK earned 250 total points
ID: 41894167
The ISO 8859-1 "Western European" coded character set does not include the Trademark symbol:

Print of grid showing ISO 8859-1 character set
The printed grid above show the characters in that character set (using the Courier typeface).
The four-character values shown at the top of each cell are the Unicode code-points; note that these are exactly the same (after excluding the leading "00" characters) as the character codes in that character set; i.e. ISO 8859-1 is an exact subset of Unicode.

To display/print the Trade Mark Sign (Unicode code-point U+2122) means that you'd have to select a different character set; this would be one of:
A different 8-bit coded-character set (which then means that some of the ISO 8859-1 characters would probably not be available, or at least not where they were expected, by standard systems.
Unicode; this reserves a unique code-point for all of the characters in all of the languages currently used in the world (and some older ones as well).

There are several different ways of representing Unicode code-points (which can range from U+0000 to U+10FFFF).
To avoid all characters having to be encoded using two (or more) bytes, Unicode is most commonly encoded using the UTF-8 transformation format;
With UTF-8, all of the first 128 characters are encoded using a single byte (which means that the UTF-8 value is the same as the ASCII value), but all other character require two or more bytes.

So, for example:

'Latin Capital Letter A', at U+0041, is encoded in UTF-8 asa single-byte (decimal) 65, or (hexadecimal) A1 value; this is the same as the ASCII character code value.
'Trade Mark Sign', at U+2122, is encoded in UTF-8 as the three bytes (hexadecimal) E284A2; if this value is decoded within a system expecting ISO 8859-1, it will probably show as something like 'â¢'.

Whatever coded character set you choose, you have to ensure that each end of a 'transaction' (and points in between) all have to know which coded character set is in use.

To avoid the possibility (in the Western world) of having to use multiple 8-bit character sets, it is best to choose UTF-8, since this is becoming the de-facto 'lingua franca'.
Asian languages have to use 16-bit character sets (e.g. Shift-JIS, GBK, etc.), because of the number of 'characters' required, but these can also be encoded using Unicode/UTF-8.

Author Closing Comment

by:Health Payment Systems
ID: 41899251
Thank you DansDad and John. This helped us decide what needed to be done.
LVL 93

Expert Comment

by:John Hurst
ID: 41899258
You are very welcome and I was happy to help.
LVL 16

Expert Comment

ID: 41899329
I've just noticed a 'typo' in my reply above; I stated:

'Latin Capital Letter A', at U+0041, is encoded in UTF-8 as a single-byte (decimal) 65, or (hexadecimal) A1 value; this is the same as the ASCII character code value.

This should have read:

'Latin Capital Letter A', at U+0041, is encoded in UTF-8 as a single-byte (decimal) 65, or (hexadecimal) 41 value; this is the same as the ASCII character code value.

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article explains how to install and use the NTBackup utility that comes with Windows Server.
How to resolve IMCEAEX NDRs in Exchange or Exchange Online related to invalid X500 addresses.
In this video we show how to create a Resource Mailbox in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: Navigate to the Recipients >> Resources tab.: "Recipients" is our default selection …
This video discusses moving either the default database or any database to a new volume.

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question