We help IT Professionals succeed at work.

Control Characters Visible on one particular system and no others

163 Views
Last Modified: 2017-11-18
We have an accounting package installed on multiple Windows servers.
  • Windows 2003 Server
  • Windows 2008 Server
  • Windows 2012 R2 Server

On our Windows 2003 Server, control characters (e.g. CR-LF) that are inadvertently entered into a character field of this accounting package are visible as vertical bars.  In the attached snippet these can be seen in the description field.  This typically happens when a user copies an Excel spreadsheet cell and pastes into the field.  We like that these control characters are visible because it alerts the user to the issue and these control characters create problems.  If we could figure out why they show up on that one box we would try to reproduce this behavior on the newer servers.

On none of the other systems, running the same version of the software, are these characters visible (although they are present).  On demonstrating this to the senior support team for the accounting package, I have learned that they have never seen this behavior on any of their servers, going back to when 2003 Server was their primary platform.  This is not an application that permits you to configure fonts or character sets.  Whatever is different seems likely to be a system-wide setting.

I have reset the Windows Display themes on the  to the Windows Classic.  But that had no effect.
Capture.JPG
Comment
Watch Question

CERTIFIED EXPERT

Commented:
Could be the system displaying the bars is using a different font for the edit control text.

This can be adjusted in control panel, color and appearance, window colors and metrics. Click on the part you want to change (I assume its "Window Text") and change the font.
Check on the working system which font its using and see if its available on the newer systems.

HTH
CERTIFIED EXPERT

Commented:
https://mcmw.abilitynet.org.uk/windows-7-changing-fonts/

Probably better than my description :)
David Johnson, CDSimple Geek from the '70s
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
This typically happens when a user copies an Excel spreadsheet cell and pastes into the field.  We like that these control characters are visible because it alerts the user to the issue and these control characters create problems Same version of the client? The allowing of ctrl characters means that the programmer is not validating/sanitizing the input from the user and allowing  non-alphanumeric input. This is a bug that they should address.

This is breaking a cardinal programming rule to never trust user input.  This means that your database is corrupted.  Imagine if you did a search for 'something' and got no results  yet search for 'something[cr][lf]' or "something||" will return results..

Author

Commented:
Adjusting the various windows fonts in the Display settings control panel does not have any impact on this issue, as least none I have tried.

Author

Commented:
I agree the input validation is not what it should be.  I have been pushing the company that developed the software to address this.  Fortunately the field involved is not heavily used in queries but the corruption does result in some issues.

Author

Commented:
Another anomaly is that on this system, masked passwords appear as vertical bars as opposed to round "bullets" typically shown.
Another anomaly is that on this system, masked passwords appear as vertical bars as opposed to round "bullets" typically shown

Another indication that there are differences in the fonts and/or coded character sets used on the different systems.

Author

Commented:
I agree.  Fonts however have been reset to defaults however.  Still at lost how to identify and adjust the character sets in play.
This problem has been solved!
(Unlock this solution with a 7-day Free Trial)
UNLOCK SOLUTION

Author

Commented:
I have followed your suggestion DansDadUK and gained the following additional insights.
It appears the display font used by this accounting package is MS San Serif.  On all systems, this font looks identical to the input field font and the corresponding control code (which I know to be 0D hex) is visible on our Windows 2003 servers (retired) as a vertical black bar.  This is how the program displays it on that OS.
 2003 Server MS Sans SerifOn the newer OS this character is invisible between the Female and Note symbols in the text file and invisible in the program.
 2008 Server MS Sans SerifI have verified I am not looking at an OpenType version of MS Sans Serif.  For the regular font, Western Script is the only one available.  I expect that my issue is that the font itself changed between versions, not that there is a different font or character set involved.  I am going to experiment with attempting to substitute the old version of the font for the new one on a test system and see what effect that has.
CERTIFIED EXPERT

Commented:
Nice finding. You could try and copy MS San Serif font from the old to the new system (make backup of existing font first)

Should be C:\Windows\fonts\sserife.fon
the corresponding control code (which I know to be 0D hex) is visible on our Windows 2003 servers (retired) as a vertical black bar.  This is how the program displays it on that OS

Looking more closely at your image, for the 'characters' in the range 0x00 - 0x0F:

extract from NotePad image
it appears that "the program" (i.e. NotePad) has shown narrow vertical rectangles for all of those bytes except:
0x00 - shown as space
0x09 - interpreted as horizontal tab

i.e. it has shown the 'bar' glyph  for the individual 0x0A (LineFeed) and 0x0D (CarriageReturn) characters.
But each line in the test file is terminated with a 0x0D0A (CarriageReturn LineFeed) pair, and these characters are not shown as bars - they are interpreted as the control codes they are.
So there is some sort of level of filtering occurring within the application.


... and just a few general comments:
Most operating systems (including Windows) now use Unicode as the 'internal' encoding.
In the Unicode coded-character set, the range U+0000 -> U+001F is reserved for the C0 control-code characters.
Similarly, the range U+0080 -> U+009F is reserved for the C1 control-code characters.
The 8-bit ISO-8859-1 "Latin-1" character set is a strict subset of Unicode; the range 0x00 --> 0xFF exactly matches U+0000 -> U+00FF.
The "Windows ANSI" (codepage 1252) character set (which is probably what matches "Western script") is a superset of ISO-8859-1 (in that it replaces some of the C1 control-code characters with Unicode graphic characters defined in ranges above U+0100), but that means that it departs from the Unicode coded character set.
Most systems now use Unicode, or one of its encoded representations (typically UTF-8), so it is probably unwise to rely on encoding which are not compliant.
Seth SimmonsLead Systems Administrator
CERTIFIED EXPERT

Commented:
No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Accept: DansDadUK (https:#a42057847)

If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

seth2740
Experts-Exchange Cleanup Volunteer

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions