Link to home
Start Free TrialLog in
Avatar of dalebrumbaugh
dalebrumbaugh

asked on

Multiple Threads Accessing the Same Data

Hello:

I have an application that creates a new thread every second.  Each thread is passed the address of a method in an instance of a class.  The method polls a directory for files.  The method then checks for a trigger file (filename.not) for each file.  If the file does not have a trigger file, no processing is done on it.  So, when the thread finds the trigger file, it deletes it, so no other thread will try to process it.  

Here's the code:

Private Sub TimerTick(ByVal sender As Object, ByVal e As System.EventArgs)
     transmission = New TransmissionThread()
     Dim t As New Threading.Thread(AddressOf transmission.MainProcess)
     t.Start()
End Sub

class transmisssion
    Public Sub MainProcess()
            For Each sFile In IO.Directory.GetFiles(reg.PGCDirectory, "*.file")
            Dim fi As New IO.FileInfo(sFile)
            If IO.File.Exists(sFile & ".not") Then
                ' Here the error occurs, when more than one thread executes the following:
                IO.File.Delete(sFile & ".not")
                Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
                bChunkCreated = True
                IO.File.Move(sFile, sTarget & fi.Name)
            End If
        Next
    End Sub
end class

My problem lies in the fact that there are instances when two or more threads are trying to delete the trigger file at the same time (which happens when you are working with 800+ files).  

I've tried setting the attribute to read-only and then having the thread check the attribute before trying to access it....however, I then run into the problem of two or more threads trying to write the attribute, or the attribute is not changed quick enough for the other thread to recognize it.

How do I communicate to the thread that a file is already being accessed by another thread?  

Thanks,
Jeremiah

Avatar of Mike Tomlinson
Mike Tomlinson
Flag of United States of America image

Why are you using a Timer and creating another thread every second?....that is an unnecessary burden on the system.

Just thread your MainProcess() and the problem will go away.

*** The code below isn't complete...it looks like you have some other variables in use that are not declared.
*** Unlesss those are Global variables?

So something like...

Public Class Transmisssion

    Private aborted As Boolean = False
    Private t As System.Threading.Thread
   
    Public Sub Start()
        If IsNothing(t) Then
            t = New System.Threading.Thread(AddressOf Me.MainProcess)
            t.Start()
        End If
    End Sub

    Public Sub Stop()
        If Not IsNothing(t) Then
            aborted = True
        End If
    End If

    Private Sub MainProcess()
        While Not aborted
            Dim sFile As String
            Dim fi As IO.FileInfo
            Dim p As Process
            For Each sFile In IO.Directory.GetFiles(reg.PGCDirectory, "*.file")
                fi = New IO.FileInfo(sFile)
                If IO.File.Exists(sFile & ".not") Then
                    ' Here the error occurs, when more than one thread executes the following:
                    IO.File.Delete(sFile & ".not")
                    Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
                    bChunkCreated = True
                    IO.File.Move(sFile, sTarget & fi.Name)
                End If
                If aborted Then
                    Exit For
                End If
            Next
         
            ' optional delay between runs...
            If Not aborted Then
                System.Threading.Thread.Sleep(1000) ' one second delay between runs
            End If        
        Wend
    End Sub

End Class
There is an extra line in there:

    Dim p As Process

You can replace your call to the legacy Shell() function using the Process class.
Or if you want multiple concurrent chunks happening, you could stay with what you had originally and use a proper thread synchronization technique instead of using a file.  Look at thread synchronization in help, and probably have your threads acquire and release a mutex while manipulating the trigger file.
Avatar of dalebrumbaugh
dalebrumbaugh

ASKER

I'm wanting multiple threads to be working on multiple file chunks.  This way one thread can be working on chunking some files while another is sending it's chunk to a destination.  I'll look at mastoo's suggestion...
mastoo...
could you give me some code examples for the "mutex" idea you mentioned?

Thanks, Jeremiah
Another reason for the multiple threads is the requirement that the application must write/send a .zip file every second regardless of whether there are any files to process or not.  So, a single thread would work until there are 800 files to zip up...then there would be a few skipped zip files for a few seconds.
It would look something like this (using ms help snippet):

For Each sFile In IO.Directory.GetFiles(reg.PGCDirectory, "*.file")
    Dim bTriggered as boolean
    Mutex* m = new Mutex(false, S"MyMutex");
    // Try to gain control of the named mutex. If the mutex is
    // controlled by another thread, wait for it to be released.        
    m->WaitOne();
    Dim fi As New IO.FileInfo(sFile)
    bTriggered = ( IO.File.Exists(sFile & ".not") )
    m->ReleaseMutex
    If bTriggered then
      IO.File.Delete(sFile & ".not")
      Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
      bChunkCreated = True
      IO.File.Move(sFile, sTarget & fi.Name)
    End If
  Next

I didn't muddy the waters with a try catch block around the mutex but you'd add that to be sure the mutext gets released if an error happens.
ASKER CERTIFIED SOLUTION
Avatar of Mike Tomlinson
Mike Tomlinson
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Whoops.  I never get it right the first time.  Idle_Mind's example would have similar trouble.  How about this:

For Each sFile In IO.Directory.GetFiles(reg.PGCDirectory, "*.file")
    Dim bTriggered as boolean = false
    Mutex* m = new Mutex(false, S"MyMutex");
    // Try to gain control of the named mutex. If the mutex is
    // controlled by another thread, wait for it to be released.        
    m->WaitOne();
    Dim fi As New IO.FileInfo(sFile)
    if IO.File.Exists(sFile & ".not") then
      bTriggered = true
      IO.File.Delete(sFile & ".not")
    end if
    m->ReleaseMutex
    If bTriggered then
      Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
      bChunkCreated = True
      IO.File.Move(sFile, sTarget & fi.Name)
    End If
  Next

of in Idle_Mind's case you would just have to hold the mutex longer.
Thanks guys...I'll try that and let you know.  Something I came up with temporarily was the following.  Let me know if you see any flaws:

Basically, a function in a vb.net module is being called to retrieve the file references within the directory.  My question is this:

Can this vb.net module function be called by two threads at the same time, or will the function handle only one request at a time?


' code within new thread
' Call the GetFreeFiles function in the vb.net module.
For Each sFile In CB.Files.GetFreeFiles
            Dim fi As New IO.FileInfo(sFile)
            Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
            bChunkCreated = True
            IO.File.Move(sFile, sTarget & fi.Name)
        Next


' A vb.net module
Namespace CB
    Module Files
        Private reg As New CB.Registry()
        Public Function GetFreeFiles() As ArrayList
            Dim sFile As String
            Dim iCount As Integer
            Dim al As New ArrayList()
            For Each sFile In IO.Directory.GetFiles(reg.PGCDirectory, "*.pgc")
                Dim fi As New IO.FileInfo(sFile)
                If IO.File.Exists(sFile & ".not") Then
                    IO.File.Delete(sFile & ".not")
                    al.Add(sFile)
                End If
                iCount += 1
                If iCount > 100 Then
                    Exit For
                End If
            Next
            Return al
        End Function
    End Module
End Namespace

Why wouldn't that suffer the same defect as your original code?

And incidentally, you could test this by playing with the threads window in the debugger.  Put a breakpoint in the function and the first thread stops, then the second thread comes in and by stepping you'll see they both are going to try to act on the same file.
The mutex solved the issue.  Thanks for all of your help guys!!