Avatar of fakaul
fakaul
Flag for United States of America asked on

Control Characters Visible on one particular system and no others

We have an accounting package installed on multiple Windows servers.
  • Windows 2003 Server
  • Windows 2008 Server
  • Windows 2012 R2 Server

On our Windows 2003 Server, control characters (e.g. CR-LF) that are inadvertently entered into a character field of this accounting package are visible as vertical bars.  In the attached snippet these can be seen in the description field.  This typically happens when a user copies an Excel spreadsheet cell and pastes into the field.  We like that these control characters are visible because it alerts the user to the issue and these control characters create problems.  If we could figure out why they show up on that one box we would try to reproduce this behavior on the newer servers.

On none of the other systems, running the same version of the software, are these characters visible (although they are present).  On demonstrating this to the senior support team for the accounting package, I have learned that they have never seen this behavior on any of their servers, going back to when 2003 Server was their primary platform.  This is not an application that permits you to configure fonts or character sets.  Whatever is different seems likely to be a system-wide setting.

I have reset the Windows Display themes on the  to the Windows Classic.  But that had no effect.
Capture.JPG
Windows Server 2003Windows Server 2012

Avatar of undefined
Last Comment
Seth Simmons

8/22/2022 - Mon
Michael Pfister

Could be the system displaying the bars is using a different font for the edit control text.

This can be adjusted in control panel, color and appearance, window colors and metrics. Click on the part you want to change (I assume its "Window Text") and change the font.
Check on the working system which font its using and see if its available on the newer systems.

HTH
Michael Pfister

https://mcmw.abilitynet.org.uk/windows-7-changing-fonts/

Probably better than my description :)
David Johnson, CD

This typically happens when a user copies an Excel spreadsheet cell and pastes into the field.  We like that these control characters are visible because it alerts the user to the issue and these control characters create problems Same version of the client? The allowing of ctrl characters means that the programmer is not validating/sanitizing the input from the user and allowing  non-alphanumeric input. This is a bug that they should address.

This is breaking a cardinal programming rule to never trust user input.  This means that your database is corrupted.  Imagine if you did a search for 'something' and got no results  yet search for 'something[cr][lf]' or "something||" will return results..
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck
fakaul

ASKER
Adjusting the various windows fonts in the Display settings control panel does not have any impact on this issue, as least none I have tried.
fakaul

ASKER
I agree the input validation is not what it should be.  I have been pushing the company that developed the software to address this.  Fortunately the field involved is not heavily used in queries but the corruption does result in some issues.
fakaul

ASKER
Another anomaly is that on this system, masked passwords appear as vertical bars as opposed to round "bullets" typically shown.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
DansDadUK

Another anomaly is that on this system, masked passwords appear as vertical bars as opposed to round "bullets" typically shown

Another indication that there are differences in the fonts and/or coded character sets used on the different systems.
fakaul

ASKER
I agree.  Fonts however have been reset to defaults however.  Still at lost how to identify and adjust the character sets in play.
ASKER CERTIFIED SOLUTION
DansDadUK

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
fakaul

ASKER
I have followed your suggestion DansDadUK and gained the following additional insights.
It appears the display font used by this accounting package is MS San Serif.  On all systems, this font looks identical to the input field font and the corresponding control code (which I know to be 0D hex) is visible on our Windows 2003 servers (retired) as a vertical black bar.  This is how the program displays it on that OS.
 2003 Server MS Sans SerifOn the newer OS this character is invisible between the Female and Note symbols in the text file and invisible in the program.
 2008 Server MS Sans SerifI have verified I am not looking at an OpenType version of MS Sans Serif.  For the regular font, Western Script is the only one available.  I expect that my issue is that the font itself changed between versions, not that there is a different font or character set involved.  I am going to experiment with attempting to substitute the old version of the font for the new one on a test system and see what effect that has.
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
Michael Pfister

Nice finding. You could try and copy MS San Serif font from the old to the new system (make backup of existing font first)

Should be C:\Windows\fonts\sserife.fon
DansDadUK

the corresponding control code (which I know to be 0D hex) is visible on our Windows 2003 servers (retired) as a vertical black bar.  This is how the program displays it on that OS

Looking more closely at your image, for the 'characters' in the range 0x00 - 0x0F:

extract from NotePad image
it appears that "the program" (i.e. NotePad) has shown narrow vertical rectangles for all of those bytes except:
0x00 - shown as space
0x09 - interpreted as horizontal tab

i.e. it has shown the 'bar' glyph  for the individual 0x0A (LineFeed) and 0x0D (CarriageReturn) characters.
But each line in the test file is terminated with a 0x0D0A (CarriageReturn LineFeed) pair, and these characters are not shown as bars - they are interpreted as the control codes they are.
So there is some sort of level of filtering occurring within the application.


... and just a few general comments:
Most operating systems (including Windows) now use Unicode as the 'internal' encoding.
In the Unicode coded-character set, the range U+0000 -> U+001F is reserved for the C0 control-code characters.
Similarly, the range U+0080 -> U+009F is reserved for the C1 control-code characters.
The 8-bit ISO-8859-1 "Latin-1" character set is a strict subset of Unicode; the range 0x00 --> 0xFF exactly matches U+0000 -> U+00FF.
The "Windows ANSI" (codepage 1252) character set (which is probably what matches "Western script") is a superset of ISO-8859-1 (in that it replaces some of the C1 control-code characters with Unicode graphic characters defined in ranges above U+0100), but that means that it departs from the Unicode coded character set.
Most systems now use Unicode, or one of its encoded representations (typically UTF-8), so it is probably unwise to rely on encoding which are not compliant.
Seth Simmons

No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Accept: DansDadUK (https:#a42057847)

If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

seth2740
Experts-Exchange Cleanup Volunteer
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.