Solved

Reading Files

Posted on 2001-08-02
13
195 Views
Last Modified: 2010-05-02
I am trying to read a text file which has no <CR> characters only <LF> characters. This means I have to read the whole file and then use VB's split function to break this <LF> delimited file into array elements which are subsequently written to a temporary file.

To read the entire file I have tried using two methods;

1. ReadTextFileContents = Input$(LOF(10), #10)

   This returns the string about 80% percent of the time. But sometimes it may not return anything. I do check to see if I have read past the end of the file and make sure it is closed. The returned string is very long. ReadTextFileContents is originally Dimensioned as a String.

and

2. Line Input #10, ReadTextFileContents

This always appears to work but this statement appears to increase CPU usage markedly over the other statement. Has anyone seen this and have a suitable and efficient solution?

Thanks.


0
Comment
Question by:athena
  • 4
  • 3
  • 2
  • +3
13 Comments
 
LVL 6

Expert Comment

by:Nitin Sontakke
ID: 6344380
Is it possible for you to tell what is the maximum file size you are expecting your code to handle.
0
 
LVL 2

Expert Comment

by:DanielBlais
ID: 6344457
try this

dim fso as variant
fim f as variant
dim result as string

Set fso = CreateObject("Scripting.FileSystemObject")
set f = Set f = fso.OpenTextFile("myfile", 1)

result = f.readall
f.close
0
 
LVL 2

Expert Comment

by:DanielBlais
ID: 6344461
oups, the line should be

Set f = fso.OpenTextFile("myfile", 1)
0
 
LVL 14

Expert Comment

by:wsh2
ID: 6344642
For speed you can't beat binary access. Assuming the file is not too large.. then read / process it all in one bite. If it is very large (say over 500,000 characters) consider CHUNKing through it.. ie. reading 100,000 characters at a time and looping until the whole file is read.

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Space$(LOF(intFreeFile))
Get intFreeFile, , strBuffer
Close intFreeFile
Dim strData() As String: strData() = Split(strBuffer, vbLf)

<----- Code End ----->

0
 

Accepted Solution

by:
Carignan earned 50 total points
ID: 6344660
Public Function GetDocumentText(ByVal FileName As String) As String
    Dim Fileno As Integer

    'Get the First available File Number
    Fileno = FreeFile
   
    ' Open the the file
    Open FileName For Binary As #Fileno
   
    'Read the content
    GetDocumentText = Input(LOF(Fileno), Fileno)
    Close Fileno
End Function


BTW dont use the Scripting File Object to open and read file because it cost alot on CPU.  It can be ok for 1 or 2 files but if your planning to open a Batch of files use the methode described here.

secondo you can you this sentence to replace your LF with crlf (it can be usefull if you needed it)
strText = Replace(strText, vbLf, vbCrLf)

Tertio: if you realy need to split line and store it separalty,  Use a Collection instead of Dynamic Array.  it is alot more faster.  but if you use a Static Array, it is ok too.  but the danger of static array is some day you will go over you max size (If your file is huge), then you will get a Out of Bound Error.

Hope this help.
0
 

Expert Comment

by:Carignan
ID: 6344774
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 

Expert Comment

by:Carignan
ID: 6344786
Good thing wsh2
Dim strData() As String
strData() = Split(strBuffer, vbLf)
this is probably fast

but replace can be usefull too because it let the whole text in one big string.  insteand of spliting it into an array.  

SO
if athena want to rewrite the file she can do:



Public Sub SaveFile1(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
    Print #FileNo, strText
    Close FileNo

End Sub

>>>> OR Depending the accepted Solution


Public Sub SaveFile2(ByRef strText As String, ByRef strFileName As String)
   
    Dim FileNo As Integer
    Dim X As Long
    FileNo = FreeFile
   
    Open strFileName For Output As FileNo
   
    For X = 0 To UBound(strData) - 1
        Print #FileNo, strData(X)
    Next X
   
    Close FileNo

End Sub

>>> and it is true to say that open in binary acces is the fastest way to open files in VB
0
 
LVL 14

Expert Comment

by:wsh2
ID: 6345004
Carignan.. Your output examples are going to add a CRLF and not the sole LF that presently exists in the Text file.

To write using Binary.. just use the same process as you did to read..

<----- Code Begin ----->

Dim intFreeFile As Integer: intFreeFile = FreeFile
Open "C:\MY_TEXT_FILE" For Binary As intFreeFile
Dim strBuffer As String: strBuffer = Join(strData(), vbLf)
Put intFreeFile, , strBuffer
Close intFreeFile

<----- Code End ----->



0
 

Expert Comment

by:Carignan
ID: 6345080
wsh2 , that a pretty good answer! athena you should use the wsh2 methode if you dont wanna replace your LF with CRLF.

but the SaveFile1 methode will not replace LF by CRLF.  in fact it wile write it as is.


it up to you athena, you have plenty of choice to do what you want.  it depend only what you do with the files contents between the time you read it and the time you write it.
0
 
LVL 8

Expert Comment

by:DennisBorg
ID: 6347244
Athena:

>This returns the string about 80% percent of the time.
>But sometimes it may not return anything.
>I do check to see if I have read past the end of the file
>and make sure it is closed. The returned string
>is very long. ReadTextFileContents is originally
>Dimensioned as a String.

I suspect that you are opening the file in INPUT mode, i.e.:

   OPEN "filename" FOR INPUT AS #10

That being the case, you should open for Binary:

   Dim hFile   As Integer
   Dim sBuff   As String
   Dim aBuff() As String

   hFile = FreeFile() ' Get next available file handle
   Open "filename" For Binary As #hFile
   sBuff = Input(LOF(hFile), hFile)
   Close #hFile

   aBuff = Split(sBuff, vbLf)


-Dennis Borg
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6350933
<<... but this statement appears to increase CPU usage markedly over the other statement. >>

That's because Line Input reads the file one byte at a time until it finds a suitable ending char. Unless you have a specific reason not to, always use Binary mode. It is the fastest way.
0
 
LVL 14

Expert Comment

by:wsh2
ID: 6361211
KDivad..
I placed that in my original comment.. <wink>. Carnigan then wholeheartedly agreed, copied it into a comment of his own and then won the question.. <smile>.
0
 
LVL 5

Expert Comment

by:KDivad
ID: 6361701
Sort of. You said it was fastest, I just added "why". So I still got you beat. <sticks tongue out>

But yes, his was basically a copy of your comment...
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
DO Loop not working 4 72
Validating VB6 Function 19 56
passing a value with stream reader AFTER a ";" 3 58
Copy a row 12 57
You can of course define an array to hold data that is of a particular type like an array of Strings to hold customer names or an array of Doubles to hold customer sales, but what do you do if you want to coordinate that data? This article describes…
Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

929 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now