Converting control characters in VB.Net


I have a text file which is the result of an export from an SQL table. At first glance the data in the file is good, however it seems littered with strange characters which are causing data to be ignored when I try and manipulate it within a VB.Net application.

I have attached a hex view of the data to demonstrate.....wherever there is an 20 surrounded by 09 (I believe the 09 is a tab), this is the cause of the issue.

How can I replace / convert the 20 with a null of even a single character space?


ABM SupportIT SupportAsked:
Who is Participating?
Randy PooleCommented:
Understandable.  I am assuming that the file is in unicode or something of that sort and you would need to handle opening the file differently.  If you open your file in Hex Edit Mode, can you paste the 4 lines of the hex data
Randy PooleCommented:
they are not strange characters, that is a tab delimited text file instead of a comma delimited.  The 20 is a space which would denote most likely a blank field, just read in the string and do a split on the string using the tab character(09)
ABM SupportIT SupportAuthor Commented:
you are indeed correct and it is a tab delimited text file. You are also correct in the way to manipulate the data as that is the method I have already used, however, the data is still missing after manipulation. The code for manipulation is as below:

Indata = sr.ReadLine

                    TestArray = Indata.Split(vbTab)

                    Varstring0 = Trim(TestArray(1))
                    VarString1 = Trim(TestArray(0))
                    VarString2 = Trim(TestArray(2))
                    VarString3 = Trim(TestArray(3))
                    VarString4 = Trim(TestArray(4))
                    VarString5 = Trim(TestArray(5))
                    VarString6 = Trim(TestArray(6))
                    VarString7 = Trim(TestArray(8))
                    VarString8 = Trim(TestArray(9))
                    VarString9 = Trim(TestArray(10))
                    VarString10 = Trim(TestArray(11))
                    VarString11 = Trim(TestArray(12))

                    Address = Trim(VarString2 & "," & VarString3 & "," & VarString4 & "," & VarString6 & "," & VarString5)

The address data, after manipulation, displays as 13-15 Finkle Street,
Kendal and the postcode, as per the hex data supplied do not display
This only happens where there are spaces like this in the data
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Randy PooleCommented:
Based upon the record you provided and your logic here is what it should be doing:
5=LA9 4AB

Open in new window

Your display for Address should show:
Is this what it is doing?
ABM SupportIT SupportAuthor Commented:
Once again you are are correct, that is what it should be showing but alas not!

I put a display in and it only displays '13-15 Finkle Street,' however it does also display a length of 38 so not really sure what is going on here! As I said previously, if all the address fields contain actual data, the code works perfectly and to be fair, the code is not what what we would call complicated!
Randy PooleCommented:
can you upload the raw data record in a file so I can take a look at it.
Randy PooleCommented:
Normally if the data is cut off, then that means their is a null, if you look at the value of VarString3 in the debugger, is it by chance a null?
Randy PooleCommented:
If this is the case after doing your split do the following:
Dim c as Integer
For c = 0 To TestArray .Length - 1
      If TestArray (c) = vbNullString Then TestArray (c) = ""

Open in new window

This will also trim all the elements so you don't need to do it when declaring above
ABM SupportIT SupportAuthor Commented:
The varstrings display as follows:

2 - 13-15 Finkle Street
3 - Blank
4 - Kendal
5 - LA9 4AB
6 - Blank

It is only when concatenated into the address variable that varstrings 4 and 5 seem to disappear! I also inserted your code and that made no difference to the address string
Randy PooleCommented:
Try this after my routine I gave you:
 Address = TestArray(2) & "," & TestArray(3) & "," & TestArray(4) & "," & TestArray(6) & "," & TestArray(5)

Open in new window

ABM SupportIT SupportAuthor Commented:
No change I'm afraid! I'm sure you agree, this shouldn't be that difficult!

I've attached a file with this record and another that works!
Randy PooleCommented:
I also need you to paste the actual sub routine so I am using exactly what you are.
ABM SupportIT SupportAuthor Commented:
I've removed reference to what you have suggeted to date and set back to my original.

Dim TestArray As String()
            Dim Indata As String
            Dim Outdata As String = ""
            Dim Address As String
            Dim Varstring0 As String
            Dim VarString1 As String
            Dim VarString2 As String
            Dim VarString3 As String
            Dim VarString4 As String
            Dim VarString5 As String
            Dim VarString6 As String
            Dim VarString7 As String
            Dim VarString8 As String
            Dim VarString9 As String
            Dim VarString10 As String
            Dim VarString11 As String
            Dim Array_Count As Integer = 0
            Dim c As Integer

            Company_ID_Array(Array_Count) = ""
            Company_Name_Array(Array_Count) = ""
            Address_Line1_Array(Array_Count) = ""
            Address_Line2_Array(Array_Count) = ""
            Address_Line3_Array(Array_Count) = ""
            Address_Line4_Array(Array_Count) = ""
            Postcode_Array(Array_Count) = ""
            Telephone_Array(Array_Count) = ""
            Status_Array(Array_Count) = ""

            Array_Count += 1

            Using sr As StreamReader = File.OpenText(frmMain.Customers_File)
                Do While sr.Peek() >= 0
                    Indata = sr.ReadLine

                    TestArray = Indata.Split(vbTab)

                    Varstring0 = TestArray(1)
                    VarString1 = TestArray(0)
                    VarString2 = TestArray(2)
                    VarString3 = TestArray(3)
                    VarString4 = TestArray(4)
                    VarString5 = TestArray(5)
                    VarString6 = TestArray(6)
                    VarString7 = TestArray(8)
                    VarString8 = TestArray(9)
                    VarString9 = TestArray(10)
                    VarString10 = TestArray(11)
                    VarString11 = TestArray(12)

                    Address = Trim(VarString2 & "," & VarString3 & "," & VarString4 & "," & VarString6 & "," & VarString5)

                    If Len(Varstring0) < 35 Then
                        Outdata = Outdata & Varstring0 & Space(35 - Len(Varstring0))
                        Outdata = Outdata & Mid(Varstring0, 1, 35)
                    End If

                    Outdata = Outdata & "|"

                    If Len(VarString1) < 10 Then
                        Outdata = Outdata & VarString1 & Space(10 - Len(VarString1))
                        Outdata = Outdata & Mid(VarString1, 1, 10)
                    End If

                    Outdata = Outdata & "|"

                    If Len(Address) < 85 Then
                        Outdata = Outdata & Address & Space(85 - Len(Address))
                        Outdata = Outdata & Mid(Address, 1, 85)
                    End If
                    Outdata = Outdata & "|"

                    Outdata = Outdata & VarString7

                    Outdata = Outdata & "|"

                    Outdata = Outdata & VarString8 & "|" & VarString9 & "|" & VarString10 & "|" & VarString11


                    Company_ID_Array(Array_Count) = Varstring0
                    Company_Name_Array(Array_Count) = VarString1
                    Address_Line1_Array(Array_Count) = VarString2
                    Address_Line2_Array(Array_Count) = VarString3
                    Address_Line3_Array(Array_Count) = VarString4
                    Address_Line4_Array(Array_Count) = VarString6
                    Postcode_Array(Array_Count) = VarString5
                    Telephone_Array(Array_Count) = VarString7
                    Status_Array(Array_Count) = VarString8
                    Client_Discount_Array(Array_Count) = VarString9
                    Client_Payment_Terms_Array(Array_Count) = VarString10
                    Client_Credit_Limit_Array(Array_Count) = VarString11

                    Array_Count += 1

                    Outdata = ""
            End Using
Randy PooleCommented:
Well, using this code and copying the text from the text file and making an ANSI file from it, I get the following for Address on the 2nd pass through is:

No I am including the text file, so download it and try it.  If it works then actually attach the file you are trying
ABM SupportIT SupportAuthor Commented:
That did actually work! The only difference betwen the files is the actual file I am using is exported from the SQL server and the one that I sent you was that same file edited using notepad and then saved as something else.

I'm not sure that I can upload the file as it is actual customer data....not sure where I stand with that.
ABM SupportIT SupportAuthor Commented:
I think you are on the right lines here! I've done some research and it looks as though it may be due to a setting on the BCP program I'm using to extract the data from the SQL table. I'm using -c which apparently is a character extract but I've got a hunch this may not be all there is!!
ABM SupportIT SupportAuthor Commented:
Thanks Randy, I believe that you have pointed me in the right direction and as such I have posted a new question, specific to the problem
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.