Solved

Reading a .CSV file with data rows having varying column counts

Posted on 2008-10-13
2
521 Views
Last Modified: 2008-10-20
I've been using this Class module for reading .csv files into my VB.Net DataTables, but I just found it will not work when the data files has rows with a varying number of columns.

The .csv file's first row has 4 columns, the second row has 5 columns, the third fow has 6 columns.

I get an error when the Sub AddRow runs since it attempts to add 5 values and first row only had 4 ??

"Cannot find column 4."

I'd like to hear your thoughts on how you would read this type of data file into a DataTable.

Thanks,
JMO9966

tblQuote_Data = fileReader.ReadFile(pathFile)

Public Function ReadFile(ByVal fileName As String) As DataTable
        ' Initialize the return values
        Dim list As New List(Of String())

        Dim table As DataTable = Nothing

        Using parser As New TextFieldParser(fileName)

            ' Setup the comma-delimited file parser.
            parser.TextFieldType = FieldType.Delimited
            parser.Delimiters = New String() {","}
            parser.HasFieldsEnclosedInQuotes = True

            While Not parser.EndOfData
                Try
                    ' Read the comma-delimited text as fields into a string array.
                    Dim input As String() = parser.ReadFields()

                    If table Is Nothing Then
                        table = Me.CreateTable(Path.GetFileName(fileName), input)
                    End If

                    If input(0).Trim <> "" Then

                        Me.AddRow(table, input)
                    Else
                        'Else send to Error table

                    End If

                Catch ex As MalformedLineException
                    ' Ignore invalid lines.
                End Try
            End While
        End Using
        Return table
    End Function

 Private Function CreateTable(ByVal name As String, ByVal input As String()) As DataTable

        Dim table As New DataTable(name)
        For index As Integer = 1 To input.Length
            table.Columns.Add("F" & index)
        Next index
        Return table
    End Function

    Private Sub AddRow(ByVal table As DataTable, ByVal input As String())

        Dim row As DataRow = table.NewRow()
        For index As Integer = 0 To table.Columns.Count
            If index < input.Length Then
                row(index) = input(index)
            End If
        Next index

        table.Rows.Add(row)
    End Sub
0
Comment
Question by:JMO9966
2 Comments
 
LVL 1

Accepted Solution

by:
Hamish_Anderson earned 500 total points
ID: 22708682
JMO, do the CSV columns have literal column names?  Ie does column 4 represent "Name" and Column 6  "Address" or similar. Or are the column names irrelevant and it is purely the order that is important?

If it is the order that is important then perhaps the structure of your table needs to be more heavily normalised.  Otherwise if there is a finite number of columns then simply create the columns, name them "col1, col2, col3" in your data table then modify the logic of your code to find the end of the csv line (this should be represented by a carriage return).
0
 

Author Comment

by:JMO9966
ID: 22762501
Thanks Harnish,

I ended up setting my For Next loop to a hard-coded number of columns since this is a static value.

The csv file does not contain a column header row.

0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

I think the Typed DataTable and Typed DataSet are very good options when working with data, but I don't like auto-generated code. First, I create an Abstract Class for my DataTables Common Code.  This class Inherits from DataTable. Also, it can …
The ECB site provides FX rates for major currencies since its inception in 1999 in the form of an XML feed. The files have the following format (reducted for brevity) (CODE) There are three files available HERE (http://www.ecb.europa.eu/stats/exch…
Along with being a a promotional video for my three-day Annielytics Dashboard Seminor, this Micro Tutorial is an intro to Google Analytics API data.
With Secure Portal Encryption, the recipient is sent a link to their email address directing them to the email laundry delivery page. From there, the recipient will be required to enter a user name and password to enter the page. Once the recipient …

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question