Followup to previous recursive folder loop - showing thumbnails - pt 6

Hi sirbounty;

Try these changes in the effort to speed up the displaying of the images to the user and if you would like to try it this is what you need to do. This change uses a worker thread to get all the files names and updates the UserImagesNames array by the amount of DisplayCount. So every full page of images will be available to the user.

Add this line at class level

    ' The Worker thread
    Private WithEvents BgWorker As New System.ComponentModel.BackgroundWorker

Remove these two lines from the Form Load event

        process()
        DisplayImages()

And add this line of code

        ' Start the background worker thread. Thread runs in DoWork event.
        BgWorker.RunWorkerAsync()

Remove the Complete Private Sub DisplayImages() subroutine and replace it with the following event handler

    ' Start the Asynchronous file search. The background thread does its work from
    ' this event.
    Private Sub BgWorker_DoWork(ByVal sender As Object, _
        ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BgWorker.DoWork

        EndWorkerThreadToolStripMenuItem.Enabled = True

        ' Holds the filepath to all the JPG.LST files in question
        Dim JPGFiles() As String
        ' Temp storage for image file names
        Dim ImageFileNames As New ArrayList
        Dim FirstTimeThrough As Boolean = True

        ' Get the string file names of all the JPG.LST files
        JPGFiles = Directory.GetFiles(strFile, "JPG.LST", SearchOption.AllDirectories)
        ' Fill the UserImagesNames array with all the image names
        For Each file As String In JPGFiles
            Using sr As New StreamReader(file)
                Dim input As String = sr.ReadToEnd()
                ImageFileNames.AddRange(input.Split(CrLf, StringSplitOptions.RemoveEmptyEntries))
            End Using
            If ImageFileNames.Count > DisplayCount - 1 Then
                UserImagesNames.AddRange(ImageFileNames.GetRange(0, DisplayCount - 1))
                ImageFileNames.RemoveRange(0, DisplayCount)
                If FirstTimeThrough Then
                    FirstTimeThrough = False
                    BgWorker.ReportProgress(0)
                End If
            End If
        Next

        If ImageFileNames.Count > 0 Then
            UserImagesNames.AddRange(ImageFileNames.GetRange(0, ImageFileNames.Count))
        End If

    End Sub

The EndWorkerThreadToolStripMenuItem assumes that you will add a way to stop the worker thread from running before it has completed because you will be ending the program. I placed this on the tool strip by that name and set its initial Enabled property to false. This is also true for the control name in the BgWorker_RunWorkerCompleted coming up next.

Add the following three event handlers to your code

    ' This is called only once just after the first DisplayCount images are available
    ' to the user
    Private Sub BgWorker_ProgressChanged(ByVal sender As Object, _
        ByVal e As System.ComponentModel.ProgressChangedEventArgs) _
        Handles BgWorker.ProgressChanged

        DisplayImages()

    End Sub

    ' The background thread calls this event just before it reaches the End Sub
    ' of the DoWork event. It is OK to access user controls in the UI from this event.
    Private Sub BgWorker_RunWorkerCompleted(ByVal sender As Object, _
        ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) _
        Handles BgWorker.RunWorkerCompleted

        EndWorkerThreadToolStripMenuItem.Enabled = False
        BgWorker.Dispose()

    End Sub

    Private Sub EndWorkerThreadToolStripMenuItem_Click(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles EndWorkerThreadToolStripMenuItem.Click

        BgWorker.CancelAsync()

    End Sub


Let me know how this works out for you.

Fernando

"Remove the Complete Private Sub DisplayImages() subroutine and replace it with the following event handler"

But we're still calling it here...

Private Sub BgWorker_ProgressChanged(ByVal sender As Object, _
        ByVal e As System.ComponentModel.ProgressChangedEventArgs) _
        Handles BgWorker.ProgressChanged

        DisplayImages()

??  

What should be in its place?

Hi sirbounty;

Here I go again. I think I need to make an appointment with my doctor to find out what is going on in my brain, but I am afraid that he will tell me that there is nothing up there. LOL.

This statement :

Remove the Complete Private Sub DisplayImages() subroutine and replace it with the following event handler

Should have been read as:

Remove the Complete process subroutine and replace it with the following event handler

So let me try it this way

Remove this code:

    Private Sub process()

        Dim JPGFiles() As String

        JPGFiles = Directory.GetFiles(strFile, _
             "JPG.LST", SearchOption.AllDirectories)

        For Each file As String In JPGFiles
            Using sr As New StreamReader(file)
                Dim input As String = sr.ReadToEnd()
                UserImagesNames.AddRange(input.Split(CrLf, _
                     StringSplitOptions.RemoveEmptyEntries))
            End Using
        Next

    End Sub

And replace it with this code.

    ' Start the Asynchronous file search. The background thread does its work from
    ' this event.
    Private Sub BgWorker_DoWork(ByVal sender As Object, _
        ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BgWorker.DoWork

        EndWorkerThreadToolStripMenuItem.Enabled = True

        ' Holds the filepath to all the JPG.LST files in question
        Dim JPGFiles() As String
        ' Temp storage for image file names
        Dim ImageFileNames As New ArrayList
        Dim FirstTimeThrough As Boolean = True

        ' Get the string file names of all the JPG.LST files
        JPGFiles = Directory.GetFiles(strFile, "JPG.LST", SearchOption.AllDirectories)
        ' Fill the UserImagesNames array with all the image names
        For Each file As String In JPGFiles
            Using sr As New StreamReader(file)
                Dim input As String = sr.ReadToEnd()
                ImageFileNames.AddRange(input.Split(CrLf, StringSplitOptions.RemoveEmptyEntries))
            End Using
            If ImageFileNames.Count > DisplayCount - 1 Then
                UserImagesNames.AddRange(ImageFileNames.GetRange(0, DisplayCount - 1))
                ImageFileNames.RemoveRange(0, DisplayCount)
                If FirstTimeThrough Then
                    FirstTimeThrough = False
                    BgWorker.ReportProgress(0)
                End If
            End If
        Next

        If ImageFileNames.Count > 0 Then
            UserImagesNames.AddRange(ImageFileNames.GetRange(0, ImageFileNames.Count))
        End If

    End Sub

And do not touch the DisplayImages subroutine.

OK, I think I got it right this time.

Fernando

Presumably I just won't see anything until it's grabbed a pageful of images?
I don't really like the bgworker trick, primarily cause I don't understand it perhaps, but it doesn't seem too 'chatty' when it's running either - thus, how do you debug it? :)

I just kicked it off - will wait a half hour and see if anything appears...thanx!

To your question, "Presumably I just won't see anything until it's grabbed a pageful of images?", The BgWorker_DoWork code get created and called from the Form Load event and starts to collect the names of the image files when it has gotten DisplayCount number of file names it transfer the names to the UserImagesNames array. When BgWorker_DoWork hits the first DisplayCount of file names it raises the event ProgressChanged which executes this event handler which displays each of the images one at a time in the FlowLayoutPanel.

    Private Sub BgWorker_ProgressChanged(ByVal sender As Object, _
        ByVal e As System.ComponentModel.ProgressChangedEventArgs) _
        Handles BgWorker.ProgressChanged

        DisplayImages()

    End Sub

To your question, "thus, how do you debug it?". You can place Console.WriteLines in the code, write to a log file, or set breakpoints in the code.

It should not take very log until you see the first set of images. So if you do not see anything in about 10 sec or so please post the file to the EE web sit so that I may look at it.

Fernando

Nothing yet... :(
I'll upload the project.

https://filedb.experts-exchange.com/incoming/ee-stuff/1263-sda.zip

Hi sirbounty;

Place these two lines in the Form Load event just before the statement, BgWorker.RunWorkerAsync().

        ' Enable ProgressChanged and Cancellation mothed
        BgWorker.WorkerReportsProgress = True
        BgWorker.WorkerSupportsCancellation = True

Also you need these events handlers in order for the solution to work.

    ' This is called only once just after the first DisplayCount images are available
    ' to the user
    Private Sub BgWorker_ProgressChanged(ByVal sender As Object, _
        ByVal e As System.ComponentModel.ProgressChangedEventArgs) _
        Handles BgWorker.ProgressChanged

        DisplayImages()

    End Sub

    ' The background thread calls this event just before it reaches the End Sub
    ' of the DoWork event. It is OK to access user controls in the UI from this event.
    Private Sub BgWorker_RunWorkerCompleted(ByVal sender As Object, _
        ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) _
        Handles BgWorker.RunWorkerCompleted

        EndWorkerThreadToolStripMenuItem.Enabled = False
        BgWorker.Dispose()

    End Sub

    Private Sub EndWorkerThreadToolStripMenuItem_Click(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles EndWorkerThreadToolStripMenuItem.Click

        BgWorker.CancelAsync()

    End Sub

Fernando

Been running for a couple minutes - nothing yet... 8 |

I paused processing and it appears to be at the point where it's building the JPG array - that usually takes a while, unless it somehow breaks out of that when DisplayCount is full?

Again you should have seen some images almost immediately. This code will read file names from JPG.LST files and when it has collected DisplayCount number of files which might be 1, 2 or 3 JPG.LST files will start showing up on the display.  

Hmm - let me try restarting it then...I till have nothing after over 10 minutes...

Did you add the code I just posted?

Still nothing...  

I will take a another look at the code you sent in the morning to see if I missed anything.

Hi sirbounty;

I went over what you upload to the web site and I made it what it should be. Compare it to what you have for frmViewe. I have run this code on my system and it does work. You should be able to copy your current project to another location and replace the frmViewer.vb file with the one I posted below.

https://filedb.experts-exchange.com/incoming/ee-stuff/1271-frmViewer.zip

I noticed that you added the EndWorkerThreadToolStripMenuItem in the Images menu item on the main menu. I would either enable Images so that you have access to the Stop Processing menu item or place it somewhere else.

Fernando

"I noticed that you added the EndWorkerThreadToolStripMenuItem in the Images menu item on the main menu. I would either enable Images so that you have access to the Stop Processing menu item or place it somewhere else."

I'm not sure what you mean by that?  Are you referring to the "Save Position" item? I haven't really fully developed that - it was more of an attempt to save the file that was currently being read, along with the position in that file, if the user decided to close out the application altogether, so they could pick back up where they left off, rather than running the full gamut a second time...

I've been running off the same instance from the other day - and have noticed that the images seem to be loading slower than they were (averaged about per second initially).  It could be simply due to the bandwidth of the server that's being loaded from...not sure.  But if it's a decrease of available resources, I may eventually want to have that Save Position item available so that I can start fresh (just something to think about - certainly a new question if it becomes too much of an issue).

I will download the latest and let you know - thanx!

appears to be a logon problem (for me anyway) - can't get to ee-stuff today... :(

can't logon today either...arg
Can you get back in to ee-stuff?

I sent an e-mail to them on the 2nd and it seems to be working today the 5th.

Okay, I grabbed the file this morning, but it seems to be responding the same way...

In looking at the code, wouldn't this line need to completely finish (i.e. reading all 9000 subfolders) before proceeding to the next line?

 JPGFiles = Directory.GetFiles(strFile, "JPG.LST", SearchOption.AllDirectories)

To your question, “In looking at the code, wouldn't this line need to completely finish (i.e. reading all 9000 subfolders) before proceeding to the next line?”. It looks at the system file table and locates the 9000 file names so in answer to your question Yes.

I put together some test code to check the speed of the above code in question. The time it take to complete this line of code, JPGFiles = Directory.GetFiles(strFile, "JPG.LST", SearchOption.AllDirectories), The results of that test will be displayed in the forms title bar in the following format Minutes:Seconds:Milliseconds.  

    ‘ Place this at class level, outside of any function or subroutine.
    Dim strTime As String = ""      ' Test Speed



    ‘ In this subroutine add all the lines that have the comment “Test Speed”

    Private Sub BgWorker_DoWork(ByVal sender As Object, _
        ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BgWorker.DoWork

        EndWorkerThreadToolStripMenuItem.Enabled = True

        ' Holds the filepath to all the JPG.LST files in question
        Dim JPGFiles() As String
        ' Temp storage for image file names
        Dim ImageFileNames As New ArrayList
        Dim FirstTimeThrough As Boolean = True

        Dim sw As New Stopwatch()' Test Speed
        sw.Stop()      ' Test Speed
        sw.Reset()      ' Test Speed
        sw.Start()      ' Test Speed

        ' Get the string file names of all the JPG.LST files
        JPGFiles = Directory.GetFiles(strFile, "JPG.LST", SearchOption.AllDirectories)
        ' Fill the UserImagesNames array with all the image names

        sw.Stop()      ' Test Speed
        strTime = sw.Elapsed.Minutes.ToString & ":" & sw.Elapsed.Seconds.ToString & ":" & _
            sw.Elapsed.Milliseconds.ToString      ' Test Speed
        BgWorker.ReportProgress(0)      ' Test Speed

        For Each file As String In JPGFiles
            Using sr As New StreamReader(file)
                Dim input As String = sr.ReadToEnd()
                ImageFileNames.AddRange(input.Split(CrLf, StringSplitOptions.RemoveEmptyEntries))
            End Using
            If ImageFileNames.Count > DisplayCount - 1 Then
                UserImagesNames.AddRange(ImageFileNames.GetRange(0, DisplayCount - 1))
                ImageFileNames.RemoveRange(0, DisplayCount)
                If FirstTimeThrough Then
                    FirstTimeThrough = False
                    BgWorker.ReportProgress(0)
                End If
            End If
        Next

        If ImageFileNames.Count > 0 Then
            UserImagesNames.AddRange(ImageFileNames.GetRange(0, ImageFileNames.Count))
        End If

    End Sub


    ‘ In this subroutine add all the lines that have the comment “Test Speed”

    Private Sub BgWorker_ProgressChanged(ByVal sender As Object, _
        ByVal e As System.ComponentModel.ProgressChangedEventArgs) _
        Handles BgWorker.ProgressChanged

        If UserImagesNames.Count > 0 Then      ' Test Speed
            DisplayImages()
        End If      ' Test Speed
        Me.Text = strTime      ' Test Speed
        Me.Refresh()      ' Test Speed

    End Sub


Let me know what results you get I would be interested.

Fernando

I apologize for the delay.
I've got this running now.  Will let you know shortly...well, when it's done anyway. :)

28:56:23

Got a chance to implement this today.
Here's what I see...not sure yet 'why', as I haven't stepped through it yet.
After approximately 25 seconds, the images started appearing at the 15th one on the first line.  My "Unable to load" image didn't appear in the first 14 (and I haven't checked yet to see if that code simply not there in this latest version).
Plus, I got two lines too many (scrollbars appeared on the right) - but I imagine I can correct that myself.
Lastly, when the Next Images menuitem became available, clicking it almost immediately retrieved the next images.
Awesome work sir - I'm very impressed!
Thank you for your dedication to helping here.  Hopefully when I go over the code, I'll understand it enough to actually learn something. :^)
Again, I appreciate your help very much!

Always glad to be able to help. Let me know what you find. ;=)