VBA To Count Number of Lines in a .CSV file

Thank you for looking at my question,

One of our processes produces a very large, badly formatted .csv file.

Is there a method to open this file, count the number of lines it contains and close it again so that I can put up a progress bar to show users how far they are through subsequent processing that needs to be carried out?

I am using a variation of the code below (which I found on EE, credit to als315) to carry out the subsequent processing
Sub read_f(filen As String)
' filen - full path to file (c:\tmp\ibm.csv)
Dim Str As String, FileIN As Integer, FileO As Integer
Dim Arr() As String
FileIN = FreeFile
Open filen For Input As #FileIN
FileO = FreeFile
Open Left(filen, Len(filen) - 4) & "_out.csv" For Output As #FileO

Do While Not EOF(FileIN)
    Line Input #FileIN, Str
    Arr = Split(Str, ";")
    If Arr(7) = "OK" Then Str = Str & ";Y"
    Print #FileO, Str
Loop
Close #FileIN
Close #FileO
End Sub

Open in new window

Any help you can offer will be greatly appreciated.
Gary CroxfordOperations Support AnalystAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

regmigrantCommented:
There is no 'count records' function in VBA I am aware of and even though I prefer to use shellscript I couldn't find one in there either.
So in similar cases I have opened the file, read through all the rows and updated a counter for all the non-blank ones, then closed it before processing. It slows down the end to end  but is the only way I could find:

Open <filenaneme> For Input As #1
 NumRec = 0
 Do While Not EOF(#1)
     Line Input #1, record
     If record <> "" Then NumRec = NumRec+1
 Loop
 Close #1

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Rgonzo1971Commented:
Hi,

pls try

Sub Macro(filen As String)


Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objTF = objFSO.OpenTextFile(Left(filen, Len(filen) - 4) & "_out.csv", 1)
strIn = objTF.readall
X = Split(strIn, vbNewLine)
NumberOfLines = UBound(X) + 1
objTF.Close

End Sub

Open in new window

Regards
1
aikimarkCommented:
a very large, badly formatted .csv file
How large?  Too large to fit into available physical memory?

In what ways is the CSV malformed?

Are the lines CrLf delimited or is some other delimiter used?

Are the lines the same length?

====================
The simplest way would be to read the entire file into a string variable, use the Split() function on the end-of-line character(s), and then use the Ubound() function to see how large the resulting array is.  Only applicable when you have enough physical memory to hold the file contents and the Split() results without paging.

You can use the FileSystemObject/TextStream to read the file, using the SkipLine method, until you reach the end.  The TextStream Line property will tell you the last line, or you could just count the number of times you invoked the SkipLine method.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

aikimarkCommented:
If the data on the last line reflects the number of lines or something meaningful about the progress you are measuring, then you can do the following:

* Open the file as Binary
* Use the Seek statement to position near the end of the file, enough to capture the entire last record
* Input the characters to the end-of-file
* Split the string
* Process/inspect the data on the last record
0
aikimarkCommented:
If the file is too large, you can read 'chunks' of the file, split the chunk, and count the 'lines'

Note: With each iteration, you have to prepend the last item of the previous iteration's Split() result to the current iteration's 'chunk'
0
Gary CroxfordOperations Support AnalystAuthor Commented:
Does the trick, thank you
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Excel

From novice to tech pro — start learning today.