Solved

Reading Files

Posted on 2001-08-02
13
192 Views
Last Modified: 2010-05-02
I am trying to read a text file which has no <CR> characters only <LF> characters. This means I have to read the whole file and then use VB's split function to break this <LF> delimited file into array elements which are subsequently written to a temporary file.

To read the entire file I have tried using two methods;

1. ReadTextFileContents = Input$(LOF(10), #10)

   This returns the string about 80% percent of the time. But sometimes it may not return anything. I do check to see if I have read past the end of the file and make sure it is closed. The returned string is very long. ReadTextFileContents is originally Dimensioned as a String.

and

2. Line Input #10, ReadTextFileContents

This always appears to work but this statement appears to increase CPU usage markedly over the other statement. Has anyone seen this and have a suitable and efficient solution?

Thanks.


0
Comment
Question by:athena
  • 4
  • 3
  • 2
  • +3
13 Comments
 
LVL 6

Expert Comment

by:Nitin Sontakke
Comment Utility
Is it possible for you to tell what is the maximum file size you are expecting your code to handle.
0
 
LVL 2

Expert Comment

by:DanielBlais
Comment Utility
try this

dim fso as variant
fim f as variant
dim result as string

Set fso = CreateObject("Scripting.FileSystemObject")
set f = Set f = fso.OpenTextFile("myfile", 1)

result = f.readall
f.close
0
 
LVL 2

Expert Comment

by:DanielBlais
Comment Utility
oups, the line should be

Set f = fso.OpenTextFile("myfile", 1)
0
 
LVL 14

Expert Comment

by:wsh2
Comment Utility
For speed you can't beat binary access. Assuming the file is not too large.. then read / process it all in one bite. If it is very large (say over 500,000 characters) consider CHUNKing through it.. ie. reading 100,000 characters at a time and looping until the whole file is read.

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Space$(LOF(intFreeFile))
Get intFreeFile, , strBuffer
Close intFreeFile
Dim strData() As String: strData() = Split(strBuffer, vbLf)

<----- Code End ----->

0
 

Accepted Solution

by:
Carignan earned 50 total points
Comment Utility
Public Function GetDocumentText(ByVal FileName As String) As String
    Dim Fileno As Integer

    'Get the First available File Number
    Fileno = FreeFile
   
    ' Open the the file
    Open FileName For Binary As #Fileno
   
    'Read the content
    GetDocumentText = Input(LOF(Fileno), Fileno)
    Close Fileno
End Function


BTW dont use the Scripting File Object to open and read file because it cost alot on CPU.  It can be ok for 1 or 2 files but if your planning to open a Batch of files use the methode described here.

secondo you can you this sentence to replace your LF with crlf (it can be usefull if you needed it)
strText = Replace(strText, vbLf, vbCrLf)

Tertio: if you realy need to split line and store it separalty,  Use a Collection instead of Dynamic Array.  it is alot more faster.  but if you use a Static Array, it is ok too.  but the danger of static array is some day you will go over you max size (If your file is huge), then you will get a Out of Bound Error.

Hope this help.
0
 

Expert Comment

by:Carignan
Comment Utility
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 

Expert Comment

by:Carignan
Comment Utility
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
 
LVL 14

Expert Comment

by:wsh2
Comment Utility
Carignan.. Your output examples are going to add a CRLF and not the sole LF that presently exists in the Text file.

To write using Binary.. just use the same process as you did to read..

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Join(strData(), vbLf)
Put intFreeFile, , strBuffer
Close intFreeFile

<----- Code End ----->



0
 

Expert Comment

by:Carignan
Comment Utility
wsh2 , that a pretty good answer! athena you should use the wsh2 methode if you dont wanna replace your LF with CRLF.

but the SaveFile1 methode will not replace LF by CRLF.  in fact it wile write it as is.


it up to you athena, you have plenty of choice to do what you want.  it depend only what you do with the files contents between the time you read it and the time you write it.
0
 
LVL 8

Expert Comment

by:DennisBorg
Comment Utility
Athena:

>This returns the string about 80% percent of the time.
>But sometimes it may not return anything.
>I do check to see if I have read past the end of the file
>and make sure it is closed. The returned string
>is very long. ReadTextFileContents is originally
>Dimensioned as a String.

I suspect that you are opening the file in INPUT mode, i.e.:

   OPEN "filename" FOR INPUT AS #10

That being the case, you should open for Binary:

   Dim hFile   As Integer
   Dim sBuff   As String
   Dim aBuff() As String

   hFile = FreeFile() ' Get next available file handle
   Open "filename" For Binary As #hFile
   sBuff = Input(LOF(hFile), hFile)
   Close #hFile

   aBuff = Split(sBuff, vbLf)


-Dennis Borg
0
 
LVL 5

Expert Comment

by:KDivad
Comment Utility
<<... but this statement appears to increase CPU usage markedly over the other statement. >>

That's because Line Input reads the file one byte at a time until it finds a suitable ending char. Unless you have a specific reason not to, always use Binary mode. It is the fastest way.
0
 
LVL 14

Expert Comment

by:wsh2
Comment Utility
KDivad..
I placed that in my original comment.. <wink>. Carnigan then wholeheartedly agreed, copied it into a comment of his own and then won the question.. <smile>.
0
 
LVL 5

Expert Comment

by:KDivad
Comment Utility
Sort of. You said it was fastest, I just added "why". So I still got you beat. <sticks tongue out>

But yes, his was basically a copy of your comment...
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

There are many ways to remove duplicate entries in an SQL or Access database. Most make you temporarily insert an ID field, make a temp table and copy data back and forth, and/or are slow. Here is an easy way in VB6 using ADO to remove duplicate row…
Introduction While answering a recent question (http://www.experts-exchange.com/Q_27402310.html) in the VB classic zone, I wrote some VB code in the (Office) VBA environment, rather than fire up my older PC.  I didn't post completely correct code o…
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
Get people started with the process of using Access VBA to control Excel using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Excel. Using automation, an Access application can laun…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now