dalebrumbaugh
asked on
Multiple Threads Accessing the Same Data
Hello:
I have an application that creates a new thread every second. Each thread is passed the address of a method in an instance of a class. The method polls a directory for files. The method then checks for a trigger file (filename.not) for each file. If the file does not have a trigger file, no processing is done on it. So, when the thread finds the trigger file, it deletes it, so no other thread will try to process it.
Here's the code:
Private Sub TimerTick(ByVal sender As Object, ByVal e As System.EventArgs)
transmission = New TransmissionThread()
Dim t As New Threading.Thread(AddressOf transmission.MainProcess)
t.Start()
End Sub
class transmisssion
Public Sub MainProcess()
For Each sFile In IO.Directory.GetFiles(reg. PGCDirecto ry, "*.file")
Dim fi As New IO.FileInfo(sFile)
If IO.File.Exists(sFile & ".not") Then
' Here the error occurs, when more than one thread executes the following:
IO.File.Delete(sFile & ".not")
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
End Sub
end class
My problem lies in the fact that there are instances when two or more threads are trying to delete the trigger file at the same time (which happens when you are working with 800+ files).
I've tried setting the attribute to read-only and then having the thread check the attribute before trying to access it....however, I then run into the problem of two or more threads trying to write the attribute, or the attribute is not changed quick enough for the other thread to recognize it.
How do I communicate to the thread that a file is already being accessed by another thread?
Thanks,
Jeremiah
I have an application that creates a new thread every second. Each thread is passed the address of a method in an instance of a class. The method polls a directory for files. The method then checks for a trigger file (filename.not) for each file. If the file does not have a trigger file, no processing is done on it. So, when the thread finds the trigger file, it deletes it, so no other thread will try to process it.
Here's the code:
Private Sub TimerTick(ByVal sender As Object, ByVal e As System.EventArgs)
transmission = New TransmissionThread()
Dim t As New Threading.Thread(AddressOf
t.Start()
End Sub
class transmisssion
Public Sub MainProcess()
For Each sFile In IO.Directory.GetFiles(reg.
Dim fi As New IO.FileInfo(sFile)
If IO.File.Exists(sFile & ".not") Then
' Here the error occurs, when more than one thread executes the following:
IO.File.Delete(sFile & ".not")
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
End Sub
end class
My problem lies in the fact that there are instances when two or more threads are trying to delete the trigger file at the same time (which happens when you are working with 800+ files).
I've tried setting the attribute to read-only and then having the thread check the attribute before trying to access it....however, I then run into the problem of two or more threads trying to write the attribute, or the attribute is not changed quick enough for the other thread to recognize it.
How do I communicate to the thread that a file is already being accessed by another thread?
Thanks,
Jeremiah
There is an extra line in there:
Dim p As Process
You can replace your call to the legacy Shell() function using the Process class.
Dim p As Process
You can replace your call to the legacy Shell() function using the Process class.
Or if you want multiple concurrent chunks happening, you could stay with what you had originally and use a proper thread synchronization technique instead of using a file. Look at thread synchronization in help, and probably have your threads acquire and release a mutex while manipulating the trigger file.
ASKER
I'm wanting multiple threads to be working on multiple file chunks. This way one thread can be working on chunking some files while another is sending it's chunk to a destination. I'll look at mastoo's suggestion...
mastoo...
could you give me some code examples for the "mutex" idea you mentioned?
Thanks, Jeremiah
mastoo...
could you give me some code examples for the "mutex" idea you mentioned?
Thanks, Jeremiah
ASKER
Another reason for the multiple threads is the requirement that the application must write/send a .zip file every second regardless of whether there are any files to process or not. So, a single thread would work until there are 800 files to zip up...then there would be a few skipped zip files for a few seconds.
It would look something like this (using ms help snippet):
For Each sFile In IO.Directory.GetFiles(reg. PGCDirecto ry, "*.file")
Dim bTriggered as boolean
Mutex* m = new Mutex(false, S"MyMutex");
// Try to gain control of the named mutex. If the mutex is
// controlled by another thread, wait for it to be released.
m->WaitOne();
Dim fi As New IO.FileInfo(sFile)
bTriggered = ( IO.File.Exists(sFile & ".not") )
m->ReleaseMutex
If bTriggered then
IO.File.Delete(sFile & ".not")
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
I didn't muddy the waters with a try catch block around the mutex but you'd add that to be sure the mutext gets released if an error happens.
For Each sFile In IO.Directory.GetFiles(reg.
Dim bTriggered as boolean
Mutex* m = new Mutex(false, S"MyMutex");
// Try to gain control of the named mutex. If the mutex is
// controlled by another thread, wait for it to be released.
m->WaitOne();
Dim fi As New IO.FileInfo(sFile)
bTriggered = ( IO.File.Exists(sFile & ".not") )
m->ReleaseMutex
If bTriggered then
IO.File.Delete(sFile & ".not")
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
I didn't muddy the waters with a try catch block around the mutex but you'd add that to be sure the mutext gets released if an error happens.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Whoops. I never get it right the first time. Idle_Mind's example would have similar trouble. How about this:
For Each sFile In IO.Directory.GetFiles(reg. PGCDirecto ry, "*.file")
Dim bTriggered as boolean = false
Mutex* m = new Mutex(false, S"MyMutex");
// Try to gain control of the named mutex. If the mutex is
// controlled by another thread, wait for it to be released.
m->WaitOne();
Dim fi As New IO.FileInfo(sFile)
if IO.File.Exists(sFile & ".not") then
bTriggered = true
IO.File.Delete(sFile & ".not")
end if
m->ReleaseMutex
If bTriggered then
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
of in Idle_Mind's case you would just have to hold the mutex longer.
For Each sFile In IO.Directory.GetFiles(reg.
Dim bTriggered as boolean = false
Mutex* m = new Mutex(false, S"MyMutex");
// Try to gain control of the named mutex. If the mutex is
// controlled by another thread, wait for it to be released.
m->WaitOne();
Dim fi As New IO.FileInfo(sFile)
if IO.File.Exists(sFile & ".not") then
bTriggered = true
IO.File.Delete(sFile & ".not")
end if
m->ReleaseMutex
If bTriggered then
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
Next
of in Idle_Mind's case you would just have to hold the mutex longer.
ASKER
Thanks guys...I'll try that and let you know. Something I came up with temporarily was the following. Let me know if you see any flaws:
Basically, a function in a vb.net module is being called to retrieve the file references within the directory. My question is this:
Can this vb.net module function be called by two threads at the same time, or will the function handle only one request at a time?
' code within new thread
' Call the GetFreeFiles function in the vb.net module.
For Each sFile In CB.Files.GetFreeFiles
Dim fi As New IO.FileInfo(sFile)
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
Next
' A vb.net module
Namespace CB
Module Files
Private reg As New CB.Registry()
Public Function GetFreeFiles() As ArrayList
Dim sFile As String
Dim iCount As Integer
Dim al As New ArrayList()
For Each sFile In IO.Directory.GetFiles(reg. PGCDirecto ry, "*.pgc")
Dim fi As New IO.FileInfo(sFile)
If IO.File.Exists(sFile & ".not") Then
IO.File.Delete(sFile & ".not")
al.Add(sFile)
End If
iCount += 1
If iCount > 100 Then
Exit For
End If
Next
Return al
End Function
End Module
End Namespace
Basically, a function in a vb.net module is being called to retrieve the file references within the directory. My question is this:
Can this vb.net module function be called by two threads at the same time, or will the function handle only one request at a time?
' code within new thread
' Call the GetFreeFiles function in the vb.net module.
For Each sFile In CB.Files.GetFreeFiles
Dim fi As New IO.FileInfo(sFile)
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
Next
' A vb.net module
Namespace CB
Module Files
Private reg As New CB.Registry()
Public Function GetFreeFiles() As ArrayList
Dim sFile As String
Dim iCount As Integer
Dim al As New ArrayList()
For Each sFile In IO.Directory.GetFiles(reg.
Dim fi As New IO.FileInfo(sFile)
If IO.File.Exists(sFile & ".not") Then
IO.File.Delete(sFile & ".not")
al.Add(sFile)
End If
iCount += 1
If iCount > 100 Then
Exit For
End If
Next
Return al
End Function
End Module
End Namespace
Why wouldn't that suffer the same defect as your original code?
And incidentally, you could test this by playing with the threads window in the debugger. Put a breakpoint in the function and the first thread stops, then the second thread comes in and by stepping you'll see they both are going to try to act on the same file.
And incidentally, you could test this by playing with the threads window in the debugger. Put a breakpoint in the function and the first thread stops, then the second thread comes in and by stepping you'll see they both are going to try to act on the same file.
ASKER
The mutex solved the issue. Thanks for all of your help guys!!
Just thread your MainProcess() and the problem will go away.
*** The code below isn't complete...it looks like you have some other variables in use that are not declared.
*** Unlesss those are Global variables?
So something like...
Public Class Transmisssion
Private aborted As Boolean = False
Private t As System.Threading.Thread
Public Sub Start()
If IsNothing(t) Then
t = New System.Threading.Thread(Ad
t.Start()
End If
End Sub
Public Sub Stop()
If Not IsNothing(t) Then
aborted = True
End If
End If
Private Sub MainProcess()
While Not aborted
Dim sFile As String
Dim fi As IO.FileInfo
Dim p As Process
For Each sFile In IO.Directory.GetFiles(reg.
fi = New IO.FileInfo(sFile)
If IO.File.Exists(sFile & ".not") Then
' Here the error occurs, when more than one thread executes the following:
IO.File.Delete(sFile & ".not")
Shell("c:\program files\winzip\WZZIP.EXE " & sTarget & sChunkName & " -a " & sFile, AppWinStyle.Hide, True, 5000)
bChunkCreated = True
IO.File.Move(sFile, sTarget & fi.Name)
End If
If aborted Then
Exit For
End If
Next
' optional delay between runs...
If Not aborted Then
System.Threading.Thread.Sl
End If
Wend
End Sub
End Class