asked on

VBA To Count Number of Lines in a .CSV file

Thank you for looking at my question,

One of our processes produces a very large, badly formatted .csv file.

Is there a method to open this file, count the number of lines it contains and close it again so that I can put up a progress bar to show users how far they are through subsequent processing that needs to be carried out?

I am using a variation of the code below (which I found on EE, credit to als315) to carry out the subsequent processing

Sub read_f(filen As String)
' filen - full path to file (c:\tmp\ibm.csv)
Dim Str As String, FileIN As Integer, FileO As Integer
Dim Arr() As String
FileIN = FreeFile
Open filen For Input As #FileIN
FileO = FreeFile
Open Left(filen, Len(filen) - 4) & "_out.csv" For Output As #FileO

Do While Not EOF(FileIN)
    Line Input #FileIN, Str
    Arr = Split(Str, ";")
    If Arr(7) = "OK" Then Str = Str & ";Y"
    Print #FileO, Str
Loop
Close #FileIN
Close #FileO
End Sub

Open in new window

Any help you can offer will be greatly appreciated.

ASKER CERTIFIED SOLUTION

regmigrant

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Rgonzo1971

Hi,

pls try

Sub Macro(filen As String)


Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objTF = objFSO.OpenTextFile(Left(filen, Len(filen) - 4) & "_out.csv", 1)
strIn = objTF.readall
X = Split(strIn, vbNewLine)
NumberOfLines = UBound(X) + 1
objTF.Close

End Sub

Open in new window

Regards

aikimark

a very large, badly formatted .csv file

How large? Too large to fit into available physical memory?

In what ways is the CSV malformed?

Are the lines CrLf delimited or is some other delimiter used?

Are the lines the same length?

====================
The simplest way would be to read the entire file into a string variable, use the Split() function on the end-of-line character(s), and then use the Ubound() function to see how large the resulting array is. Only applicable when you have enough physical memory to hold the file contents and the Split() results without paging.

You can use the FileSystemObject/TextStream to read the file, using the SkipLine method, until you reach the end. The TextStream Line property will tell you the last line, or you could just count the number of times you invoked the SkipLine method.

aikimark

If the data on the last line reflects the number of lines or something meaningful about the progress you are measuring, then you can do the following:

* Open the file as Binary
* Use the Seek statement to position near the end of the file, enough to capture the entire last record
* Input the characters to the end-of-file
* Split the string
* Process/inspect the data on the last record

aikimark

If the file is too large, you can read 'chunks' of the file, split the chunk, and count the 'lines'

Note: With each iteration, you have to prepend the last item of the previous iteration's Split() result to the current iteration's 'chunk'

Gary Croxford

ASKER

Does the trick, thank you