Solved

Reading Files

Posted on 2001-08-02
13
204 Views
Last Modified: 2010-05-02
I am trying to read a text file which has no <CR> characters only <LF> characters. This means I have to read the whole file and then use VB's split function to break this <LF> delimited file into array elements which are subsequently written to a temporary file.

To read the entire file I have tried using two methods;

1. ReadTextFileContents = Input$(LOF(10), #10)

   This returns the string about 80% percent of the time. But sometimes it may not return anything. I do check to see if I have read past the end of the file and make sure it is closed. The returned string is very long. ReadTextFileContents is originally Dimensioned as a String.

and

2. Line Input #10, ReadTextFileContents

This always appears to work but this statement appears to increase CPU usage markedly over the other statement. Has anyone seen this and have a suitable and efficient solution?

Thanks.


0
Comment
Question by:athena
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +3
13 Comments
 
LVL 9

Expert Comment

by:Nitin Sontakke
ID: 6344380
Is it possible for you to tell what is the maximum file size you are expecting your code to handle.
0
 
LVL 2

Expert Comment

by:DanielBlais
ID: 6344457
try this

dim fso as variant
fim f as variant
dim result as string

Set fso = CreateObject("Scripting.FileSystemObject")
set f = Set f = fso.OpenTextFile("myfile", 1)

result = f.readall
f.close
0
 
LVL 2

Expert Comment

by:DanielBlais
ID: 6344461
oups, the line should be

Set f = fso.OpenTextFile("myfile", 1)
0
Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

 
LVL 14

Expert Comment

by:wsh2
ID: 6344642
For speed you can't beat binary access. Assuming the file is not too large.. then read / process it all in one bite. If it is very large (say over 500,000 characters) consider CHUNKing through it.. ie. reading 100,000 characters at a time and looping until the whole file is read.

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Space$(LOF(intFreeFile))
Get intFreeFile, , strBuffer
Close intFreeFile
Dim strData() As String: strData() = Split(strBuffer, vbLf)

<----- Code End ----->

0
 

Accepted Solution

by:
Carignan earned 50 total points
ID: 6344660
Public Function GetDocumentText(ByVal FileName As String) As String
    Dim Fileno As Integer

    'Get the First available File Number
    Fileno = FreeFile
   
    ' Open the the file
    Open FileName For Binary As #Fileno
   
    'Read the content
    GetDocumentText = Input(LOF(Fileno), Fileno)
    Close Fileno
End Function


BTW dont use the Scripting File Object to open and read file because it cost alot on CPU.  It can be ok for 1 or 2 files but if your planning to open a Batch of files use the methode described here.

secondo you can you this sentence to replace your LF with crlf (it can be usefull if you needed it)
strText = Replace(strText, vbLf, vbCrLf)

Tertio: if you realy need to split line and store it separalty,  Use a Collection instead of Dynamic Array.  it is alot more faster.  but if you use a Static Array, it is ok too.  but the danger of static array is some day you will go over you max size (If your file is huge), then you will get a Out of Bound Error.

Hope this help.
0
 

Expert Comment

by:Carignan
ID: 6344774
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
 

Expert Comment

by:Carignan
ID: 6344786
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
 
LVL 14

Expert Comment

by:wsh2
ID: 6345004
Carignan.. Your output examples are going to add a CRLF and not the sole LF that presently exists in the Text file.

To write using Binary.. just use the same process as you did to read..

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Join(strData(), vbLf)
Put intFreeFile, , strBuffer
Close intFreeFile

<----- Code End ----->



0
 

Expert Comment

by:Carignan
ID: 6345080
wsh2 , that a pretty good answer! athena you should use the wsh2 methode if you dont wanna replace your LF with CRLF.

but the SaveFile1 methode will not replace LF by CRLF.  in fact it wile write it as is.


it up to you athena, you have plenty of choice to do what you want.  it depend only what you do with the files contents between the time you read it and the time you write it.
0
 
LVL 8

Expert Comment

by:DennisBorg
ID: 6347244
Athena:

>This returns the string about 80% percent of the time.
>But sometimes it may not return anything.
>I do check to see if I have read past the end of the file
>and make sure it is closed. The returned string
>is very long. ReadTextFileContents is originally
>Dimensioned as a String.

I suspect that you are opening the file in INPUT mode, i.e.:

   OPEN "filename" FOR INPUT AS #10

That being the case, you should open for Binary:

   Dim hFile   As Integer
   Dim sBuff   As String
   Dim aBuff() As String

   hFile = FreeFile() ' Get next available file handle
   Open "filename" For Binary As #hFile
   sBuff = Input(LOF(hFile), hFile)
   Close #hFile

   aBuff = Split(sBuff, vbLf)


-Dennis Borg
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6350933
<<... but this statement appears to increase CPU usage markedly over the other statement. >>

That's because Line Input reads the file one byte at a time until it finds a suitable ending char. Unless you have a specific reason not to, always use Binary mode. It is the fastest way.
0
 
LVL 14

Expert Comment

by:wsh2
ID: 6361211
KDivad..
I placed that in my original comment.. <wink>. Carnigan then wholeheartedly agreed, copied it into a comment of his own and then won the question.. <smile>.
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6361701
Sort of. You said it was fastest, I just added "why". So I still got you beat. <sticks tongue out>

But yes, his was basically a copy of your comment...
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction I needed to skip over some file processing within a For...Next loop in some old production code and wished that VB (classic) had a statement that would drop down to the end of the current iteration, bypassing the statements that were c…
I’ve seen a number of people looking for examples of how to access web services from VB6.  I’ve been using a test harness I built in VB6 (using many resources I found online) that I use for small projects to work out how to communicate with web serv…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question