Reading Files

I am trying to read a text file which has no <CR> characters only <LF> characters. This means I have to read the whole file and then use VB's split function to break this <LF> delimited file into array elements which are subsequently written to a temporary file.

To read the entire file I have tried using two methods;

1. ReadTextFileContents = Input$(LOF(10), #10)

   This returns the string about 80% percent of the time. But sometimes it may not return anything. I do check to see if I have read past the end of the file and make sure it is closed. The returned string is very long. ReadTextFileContents is originally Dimensioned as a String.

and

2. Line Input #10, ReadTextFileContents

This always appears to work but this statement appears to increase CPU usage markedly over the other statement. Has anyone seen this and have a suitable and efficient solution?

Thanks.


athenaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Nitin SontakkeDeveloperCommented:
Is it possible for you to tell what is the maximum file size you are expecting your code to handle.
0
DanielBlaisCommented:
try this

dim fso as variant
fim f as variant
dim result as string

Set fso = CreateObject("Scripting.FileSystemObject")
set f = Set f = fso.OpenTextFile("myfile", 1)

result = f.readall
f.close
0
DanielBlaisCommented:
oups, the line should be

Set f = fso.OpenTextFile("myfile", 1)
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

wsh2Commented:
For speed you can't beat binary access. Assuming the file is not too large.. then read / process it all in one bite. If it is very large (say over 500,000 characters) consider CHUNKing through it.. ie. reading 100,000 characters at a time and looping until the whole file is read.

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Space$(LOF(intFreeFile))
Get intFreeFile, , strBuffer
Close intFreeFile
Dim strData() As String: strData() = Split(strBuffer, vbLf)

<----- Code End ----->

0
CarignanCommented:
Public Function GetDocumentText(ByVal FileName As String) As String
    Dim Fileno As Integer

    'Get the First available File Number
    Fileno = FreeFile
   
    ' Open the the file
    Open FileName For Binary As #Fileno
   
    'Read the content
    GetDocumentText = Input(LOF(Fileno), Fileno)
    Close Fileno
End Function


BTW dont use the Scripting File Object to open and read file because it cost alot on CPU.  It can be ok for 1 or 2 files but if your planning to open a Batch of files use the methode described here.

secondo you can you this sentence to replace your LF with crlf (it can be usefull if you needed it)
strText = Replace(strText, vbLf, vbCrLf)

Tertio: if you realy need to split line and store it separalty,  Use a Collection instead of Dynamic Array.  it is alot more faster.  but if you use a Static Array, it is ok too.  but the danger of static array is some day you will go over you max size (If your file is huge), then you will get a Out of Bound Error.

Hope this help.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CarignanCommented:
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
CarignanCommented:
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
wsh2Commented:
Carignan.. Your output examples are going to add a CRLF and not the sole LF that presently exists in the Text file.

To write using Binary.. just use the same process as you did to read..

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Join(strData(), vbLf)
Put intFreeFile, , strBuffer
Close intFreeFile

<----- Code End ----->



0
CarignanCommented:
wsh2 , that a pretty good answer! athena you should use the wsh2 methode if you dont wanna replace your LF with CRLF.

but the SaveFile1 methode will not replace LF by CRLF.  in fact it wile write it as is.


it up to you athena, you have plenty of choice to do what you want.  it depend only what you do with the files contents between the time you read it and the time you write it.
0
DennisBorgCommented:
Athena:

>This returns the string about 80% percent of the time.
>But sometimes it may not return anything.
>I do check to see if I have read past the end of the file
>and make sure it is closed. The returned string
>is very long. ReadTextFileContents is originally
>Dimensioned as a String.

I suspect that you are opening the file in INPUT mode, i.e.:

   OPEN "filename" FOR INPUT AS #10

That being the case, you should open for Binary:

   Dim hFile   As Integer
   Dim sBuff   As String
   Dim aBuff() As String

   hFile = FreeFile() ' Get next available file handle
   Open "filename" For Binary As #hFile
   sBuff = Input(LOF(hFile), hFile)
   Close #hFile

   aBuff = Split(sBuff, vbLf)


-Dennis Borg
0
KDivadCommented:
<<... but this statement appears to increase CPU usage markedly over the other statement. >>

That's because Line Input reads the file one byte at a time until it finds a suitable ending char. Unless you have a specific reason not to, always use Binary mode. It is the fastest way.
0
wsh2Commented:
KDivad..
I placed that in my original comment.. <wink>. Carnigan then wholeheartedly agreed, copied it into a comment of his own and then won the question.. <smile>.
0
KDivadCommented:
Sort of. You said it was fastest, I just added "why". So I still got you beat. <sticks tongue out>

But yes, his was basically a copy of your comment...
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic Classic

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.