Converting control characters in VB.Net

Posted on 2014-07-22
Last Modified: 2014-07-25

I have a text file which is the result of an export from an SQL table. At first glance the data in the file is good, however it seems littered with strange characters which are causing data to be ignored when I try and manipulate it within a VB.Net application.

I have attached a hex view of the data to demonstrate.....wherever there is an 20 surrounded by 09 (I believe the 09 is a tab), this is the cause of the issue.

How can I replace / convert the 20 with a null of even a single character space?


Question by:abmcsltd
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 8
LVL 21

Expert Comment

by:Randy Poole
ID: 40211456
they are not strange characters, that is a tab delimited text file instead of a comma delimited.  The 20 is a space which would denote most likely a blank field, just read in the string and do a split on the string using the tab character(09)

Author Comment

ID: 40211565
you are indeed correct and it is a tab delimited text file. You are also correct in the way to manipulate the data as that is the method I have already used, however, the data is still missing after manipulation. The code for manipulation is as below:

Indata = sr.ReadLine

                    TestArray = Indata.Split(vbTab)

                    Varstring0 = Trim(TestArray(1))
                    VarString1 = Trim(TestArray(0))
                    VarString2 = Trim(TestArray(2))
                    VarString3 = Trim(TestArray(3))
                    VarString4 = Trim(TestArray(4))
                    VarString5 = Trim(TestArray(5))
                    VarString6 = Trim(TestArray(6))
                    VarString7 = Trim(TestArray(8))
                    VarString8 = Trim(TestArray(9))
                    VarString9 = Trim(TestArray(10))
                    VarString10 = Trim(TestArray(11))
                    VarString11 = Trim(TestArray(12))

                    Address = Trim(VarString2 & "," & VarString3 & "," & VarString4 & "," & VarString6 & "," & VarString5)

The address data, after manipulation, displays as 13-15 Finkle Street,
Kendal and the postcode, as per the hex data supplied do not display
This only happens where there are spaces like this in the data
LVL 21

Expert Comment

by:Randy Poole
ID: 40211605
Based upon the record you provided and your logic here is what it should be doing:
5=LA9 4AB

Open in new window

Your display for Address should show:
Is this what it is doing?
[Live Webinar] The Cloud Skills Gap

As Cloud technologies come of age, business leaders grapple with the impact it has on their team's skills and the gap associated with the use of a cloud platform.

Join experts from 451 Research and Concerto Cloud Services on July 27th where we will examine fact and fiction.


Author Comment

ID: 40211636
Once again you are are correct, that is what it should be showing but alas not!

I put a display in and it only displays '13-15 Finkle Street,' however it does also display a length of 38 so not really sure what is going on here! As I said previously, if all the address fields contain actual data, the code works perfectly and to be fair, the code is not what what we would call complicated!
LVL 21

Expert Comment

by:Randy Poole
ID: 40211667
can you upload the raw data record in a file so I can take a look at it.
LVL 21

Expert Comment

by:Randy Poole
ID: 40211672
Normally if the data is cut off, then that means their is a null, if you look at the value of VarString3 in the debugger, is it by chance a null?
LVL 21

Expert Comment

by:Randy Poole
ID: 40211695
If this is the case after doing your split do the following:
Dim c as Integer
For c = 0 To TestArray .Length - 1
      If TestArray (c) = vbNullString Then TestArray (c) = ""

Open in new window

This will also trim all the elements so you don't need to do it when declaring above

Author Comment

ID: 40211718
The varstrings display as follows:

2 - 13-15 Finkle Street
3 - Blank
4 - Kendal
5 - LA9 4AB
6 - Blank

It is only when concatenated into the address variable that varstrings 4 and 5 seem to disappear! I also inserted your code and that made no difference to the address string
LVL 21

Expert Comment

by:Randy Poole
ID: 40211780
Try this after my routine I gave you:
 Address = TestArray(2) & "," & TestArray(3) & "," & TestArray(4) & "," & TestArray(6) & "," & TestArray(5)

Open in new window


Author Comment

ID: 40211843
No change I'm afraid! I'm sure you agree, this shouldn't be that difficult!

I've attached a file with this record and another that works!
LVL 21

Expert Comment

by:Randy Poole
ID: 40211886
I also need you to paste the actual sub routine so I am using exactly what you are.

Author Comment

ID: 40211925
I've removed reference to what you have suggeted to date and set back to my original.

Dim TestArray As String()
            Dim Indata As String
            Dim Outdata As String = ""
            Dim Address As String
            Dim Varstring0 As String
            Dim VarString1 As String
            Dim VarString2 As String
            Dim VarString3 As String
            Dim VarString4 As String
            Dim VarString5 As String
            Dim VarString6 As String
            Dim VarString7 As String
            Dim VarString8 As String
            Dim VarString9 As String
            Dim VarString10 As String
            Dim VarString11 As String
            Dim Array_Count As Integer = 0
            Dim c As Integer

            Company_ID_Array(Array_Count) = ""
            Company_Name_Array(Array_Count) = ""
            Address_Line1_Array(Array_Count) = ""
            Address_Line2_Array(Array_Count) = ""
            Address_Line3_Array(Array_Count) = ""
            Address_Line4_Array(Array_Count) = ""
            Postcode_Array(Array_Count) = ""
            Telephone_Array(Array_Count) = ""
            Status_Array(Array_Count) = ""

            Array_Count += 1

            Using sr As StreamReader = File.OpenText(frmMain.Customers_File)
                Do While sr.Peek() >= 0
                    Indata = sr.ReadLine

                    TestArray = Indata.Split(vbTab)

                    Varstring0 = TestArray(1)
                    VarString1 = TestArray(0)
                    VarString2 = TestArray(2)
                    VarString3 = TestArray(3)
                    VarString4 = TestArray(4)
                    VarString5 = TestArray(5)
                    VarString6 = TestArray(6)
                    VarString7 = TestArray(8)
                    VarString8 = TestArray(9)
                    VarString9 = TestArray(10)
                    VarString10 = TestArray(11)
                    VarString11 = TestArray(12)

                    Address = Trim(VarString2 & "," & VarString3 & "," & VarString4 & "," & VarString6 & "," & VarString5)

                    If Len(Varstring0) < 35 Then
                        Outdata = Outdata & Varstring0 & Space(35 - Len(Varstring0))
                        Outdata = Outdata & Mid(Varstring0, 1, 35)
                    End If

                    Outdata = Outdata & "|"

                    If Len(VarString1) < 10 Then
                        Outdata = Outdata & VarString1 & Space(10 - Len(VarString1))
                        Outdata = Outdata & Mid(VarString1, 1, 10)
                    End If

                    Outdata = Outdata & "|"

                    If Len(Address) < 85 Then
                        Outdata = Outdata & Address & Space(85 - Len(Address))
                        Outdata = Outdata & Mid(Address, 1, 85)
                    End If
                    Outdata = Outdata & "|"

                    Outdata = Outdata & VarString7

                    Outdata = Outdata & "|"

                    Outdata = Outdata & VarString8 & "|" & VarString9 & "|" & VarString10 & "|" & VarString11


                    Company_ID_Array(Array_Count) = Varstring0
                    Company_Name_Array(Array_Count) = VarString1
                    Address_Line1_Array(Array_Count) = VarString2
                    Address_Line2_Array(Array_Count) = VarString3
                    Address_Line3_Array(Array_Count) = VarString4
                    Address_Line4_Array(Array_Count) = VarString6
                    Postcode_Array(Array_Count) = VarString5
                    Telephone_Array(Array_Count) = VarString7
                    Status_Array(Array_Count) = VarString8
                    Client_Discount_Array(Array_Count) = VarString9
                    Client_Payment_Terms_Array(Array_Count) = VarString10
                    Client_Credit_Limit_Array(Array_Count) = VarString11

                    Array_Count += 1

                    Outdata = ""
            End Using
LVL 21

Expert Comment

by:Randy Poole
ID: 40211986
Well, using this code and copying the text from the text file and making an ANSI file from it, I get the following for Address on the 2nd pass through is:

No I am including the text file, so download it and try it.  If it works then actually attach the file you are trying

Author Comment

ID: 40212047
That did actually work! The only difference betwen the files is the actual file I am using is exported from the SQL server and the one that I sent you was that same file edited using notepad and then saved as something else.

I'm not sure that I can upload the file as it is actual customer data....not sure where I stand with that.
LVL 21

Accepted Solution

Randy Poole earned 500 total points
ID: 40212064
Understandable.  I am assuming that the file is in unicode or something of that sort and you would need to handle opening the file differently.  If you open your file in Hex Edit Mode, can you paste the 4 lines of the hex data

Author Comment

ID: 40214125
I think you are on the right lines here! I've done some research and it looks as though it may be due to a setting on the BCP program I'm using to extract the data from the SQL table. I'm using -c which apparently is a character extract but I've got a hunch this may not be all there is!!

Author Closing Comment

ID: 40219415
Thanks Randy, I believe that you have pointed me in the right direction and as such I have posted a new question, specific to the problem

Featured Post

Salesforce Made Easy to Use

On-screen guidance at the moment of need enables you & your employees to focus on the core, you can now boost your adoption rates swiftly and simply with one easy tool.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
Computer science students often experience many of the same frustrations when going through their engineering courses. This article presents seven tips I found useful when completing a bachelors and masters degree in computing which I believe may he…
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA.…
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to go…

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question