Many Lists

Hi,
I'm writing a program, the way it needs to work is, i need to have a listbox with a list of text files...and in each text file there is about 130,000 lines. I need to have a way to go through each item in the file then go on to the next file, but the thing is, i cant just do a loop and go that way, i have to be able to get the next item in random places in the program, so i was thinking a function but i cant think of a way to do this, because it would be innefective to jsut keep openin gthe file and scanning to the place in the file to get the next item...can anyone help me out with this?
LVL 19
BrianGEFF719Asked:
Who is Participating?
 
PaulHewsCommented:
You might get it to work, but it's not very efficient to load a list for this purpose.  Controls are slower than variables, and in this case, you are accessing the listbox 3 times for each of the 100K+ lines.  Once to add the item, once to process it and once to remove it.  And as Eddy pointed out, the listbox gets slower the more stuff you add to it.  It's just not designed to efficiently handle large numbers of items.  Add to that the complexity of arranging exactly 30000 lines at a time (how do you read in the lines in the first place?) and you have a nasty/inefficient/slow system that you will not be proud of, and that will be difficult for you to debug and for others to understand.

Eddy's method will be quite fast.  The only danger when reading in large amounts of data into memory is that you can have memory problems if the system is low on memory, or if the files are particularly large.  If those are not a concern then this is the way to go.

My method will be slower by comparison (and depending what kind of processing you are doing on each line, it may take quite a long time) but it doesn't read everything into memory at one time.  In programming, this is the compromise you usually make: use more memory and get more speed.

There is another in method you could use, but I wouldn't suggest it until you have more programming experience.  It involves using binary file methods to read the file in fairly large chunks, splitting those chunks into lines and processing each chunk in turn until the file is finished.  This is a compromise between the first two methods; not blowing out the memory and not wasting time in many small disk reads.


0
 
EDDYKTCommented:
I think you can read the whole file at one shot and then use instr to find your str


ie.
Dim ff As Integer, Filename As String, s As String
   
    Filename = "your file"
    ff = FreeFile
    Open Filename For Binary Access Read Lock Read Write As #ff
    s = Space$(LOF(ff))
    Get #ff, , s
    Close ff
0
 
PaulHewsCommented:
>be able to get the next item in random places in the program

What are the rules to abandon the file we have open and go to the next one?
0
Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

 
BrianGEFF719Author Commented:
here is what i was thinking, just load 30,000 items at a time to another listbox, then when the listbox that contains the items gets empty load up another 30,000 until we hit the end of the lsit, then goto the next list...but it just doesnt seem effective to me. AND ITS NOT RANDOM ITS EACH LINE ONE AFTER ANOTHER
0
 
BrianGEFF719Author Commented:
I have not received an answer for my question, I'm going to delete it. If anyone has any last comments that they think may help please post them and i wont delete the question.


-Brian
0
 
MoondancerCommented:
Listening for updates, and hope that Experts respond here with help for you.

We are doing to do some server transitioning today and may experience a little down time.

Moondancer - EE Moderator
0
 
PaulHewsCommented:
Brian, I am still trying to understand your problem.  It sounds as if you want to open each file in sequential mode, read each line and process it.  A structured way to do that would be with a loop:

    Dim i As Integer
    Dim hFile As Integer
    Dim strLine As String
    For i = 0 To lstFiles.ListCount - 1
        hFile = FreeFile
        Open lstFiles.List(i) For Input As #hFile
        Do While Not EOF(hFile)
            Line Input #hFile, strLine
            'code to process line
        Loop
    Next i

So why isn't this approach suitable?
0
 
PaulHewsCommented:
Forgot to close:

    Dim i As Integer
    Dim hFile As Integer
    Dim strLine As String
    For i = 0 To lstFiles.ListCount - 1
        hFile = FreeFile
        Open lstFiles.List(i) For Input As #hFile
        Do While Not EOF(hFile)
            Line Input #hFile, strLine
            'code to process line
        Loop
        Close #hFile
    Next i
0
 
EDDYKTCommented:
As what I said, you can read the whole file in one shot and then use split function to separate each line.


ie.
dim arr, i as long

arr = split(s, vbcrlf)

for i=0 to ubound(arr)-1

' process your string
'arr(i)
next

Another thing I want to point out is listbox is slow if you add more than 30000 items, use listview

for more info
http://www.experts-exchange.com/jsp/qManageQuestion.jsp?ta=visualbasic&qid=20299351
0
 
BrianGEFF719Author Commented:
here is what i was thinkin, write a sub to load list items to a INVISIBLE listbox, and ill load 30,000 items at a time and when the list gets empty, it will just call the sub again, and the sub will ahve the place it left off as a static....do u ppl think that would work?
0
 
DanRollinsCommented:
Hi BrianGEFF719,
It appears that you have forgotten this question. I will ask Community Support to close it unless you finalize it within 7 days. I will ask a Community Support Moderator to:

    Accept PaulHews's comment(s) as an answer.

BrianGEFF719, if you think your question was not answered at all or if you need help, just post a new comment here; Community Support will help you.  DO NOT accept this comment as an answer.

EXPERTS: If you disagree with that recommendation, please post an explanatory comment.
==========
DanRollins -- EE database cleanup volunteer
0
 
SpideyModCommented:
per recommendation

SpideyMod
Community Support Moderator @Experts Exchange
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.