VB.net Service - File In Use

OK, I need some help with this one and the best approach to take.

I have a service that looks for NEW ZIP Files every minute...the problem I seem to be having is once it finds a new file(s) I loop through them all unzip them an process the files inside...however that processing doesn't seem to be finishing before my one minute is up and it tries to pick up those files and process them again.

How can I have a service and only look for new files?

Mark
smithmrkAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Éric MoreauSenior .Net ConsultantCommented:
MlandaTCommented:
You need to wait for the process to finish writing to the file. FileSystemWatcher is an excellent mechanism to monitor a folder and it will pick up files as soon as they are created. However, the process creating the file might still be in the process of writing the file (as you have observed). A solution I've used in the past was to monitor the Last  Modified Date on the file and only then process the if it hasn't changed in say, 5 minutes or something. This worked quite well. However, it's also important to think about moving processed files out of the folder, so that whenever your service starts, you can safely assume that anything still in the folder is new and ready to be processed.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
MlandaTCommented:
What are you using for a timer? Make sure to use the system.threading.timer, and wrap your non-reentrant code in a Monitor.TryEnter/Exit like this. Even if the timer fires,  it will not overlap processing
object lockObject = new object();

private void ProcessTimerEvent(object state) 
 {
   if (Monitor.TryEnter(lockObject))
   {
     try
     {
       // Work here
     }
     finally
     {
       Monitor.Exit(lockObject);
     }
   }
 }

Open in new window

Learn Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

it_saigeDeveloperCommented:
Assuming you are using a FileSystemWatcher and looking for the Created event, the first thing you can do is move the logic for processing the zip outside of your Created event handler.

Ideally, you would have a collection (queue for example) set aside to which you can add process-able items.  When items get added to this collection you would then fire another event to process items in the collection or have a background thread/task/worker constantly checking the queue and acting upon items that are in the queue.

As items are processed you perform clean-up to ensure that the item is no longer processable (probably by way of deletion).

An extremely simple example implementation would look something like:
Imports System.ComponentModel
Imports System.IO
Imports System.Linq
Imports System.Threading

Module Module1
	WithEvents fWatcher As New FileSystemWatcher("C:\_admin\", "*.zip") With {.EnableRaisingEvents = True, .IncludeSubdirectories = True}
	WithEvents fWorker As New BackgroundWorker() With {.WorkerReportsProgress = True, .WorkerSupportsCancellation = True}
	Private fLastChangedTime As DateTime = DateTime.Now
	Private fLastFilePath As String = String.Empty
	Private fIsRunning As Boolean = False
	Private ReadOnly QueuedFiles As New List(Of ProcessableFile)

	Sub Main()
		fIsRunning = True
		fWorker.RunWorkerAsync()
		Console.WriteLine("Press any key to exit")
		Console.ReadLine()
		fIsRunning = False
		fWorker.CancelAsync()
	End Sub

	Sub OnFileChanged(ByVal sender As Object, ByVal e As FileSystemEventArgs) Handles fWatcher.Created, fWatcher.Deleted
		Try
			Dim span = DateTime.Now.Subtract(fLastChangedTime)
			If span.TotalSeconds > 2 OrElse fLastFilePath <> e.FullPath Then
				' Wait a second for any locks to be released before taking actions
				Thread.Sleep(1000)

				Select Case e.ChangeType
					Case WatcherChangeTypes.Created
						fLastChangedTime = DateTime.Now
						fLastFilePath = e.FullPath
						Dim match = (From pFile In QueuedFiles Where pFile.File.FullName.Equals(e.FullPath, StringComparison.OrdinalIgnoreCase) Select pFile).SingleOrDefault()
						If match Is Nothing Then
							SyncLock (QueuedFiles)
								QueuedFiles.Add(New ProcessableFile(New FileInfo(e.FullPath)))
							End SyncLock
						End If
						Exit Select
					Case WatcherChangeTypes.Deleted
						fLastChangedTime = DateTime.Now
						fLastFilePath = e.FullPath
						Console.WriteLine("{0} has been deleted.", e.FullPath)
						Dim match = (From pFile In QueuedFiles Where pFile.File.FullName.Equals(e.FullPath, StringComparison.OrdinalIgnoreCase) Select pFile).SingleOrDefault()
						If match IsNot Nothing Then
							SyncLock (QueuedFiles)
								QueuedFiles.Remove(match)
								Console.WriteLine("{0} has been removed from the list.", e.FullPath)
							End SyncLock
						End If
						Exit Select
				End Select
			End If
		Catch ex As Exception
			Console.WriteLine("Exception reported - {0}", ex)
		End Try
	End Sub

	Sub OnDoWork(ByVal sender As Object, ByVal e As DoWorkEventArgs) Handles fWorker.DoWork
		Try
			While (fIsRunning)
				SyncLock (QueuedFiles)
					For i As Integer = QueuedFiles.Count - 1 To 0 Step -1
						If Not QueuedFiles(i).IsProcessing Then
							'Simulating a long process
							Thread.Sleep(10000)
							Console.WriteLine("Processing new item - {0}", QueuedFiles(i).File.FullName)
							QueuedFiles(i).IsProcessing = True
						Else
							If Not QueuedFiles(i).IsProcessed Then
								Console.WriteLine("Finished processing - {0}", QueuedFiles(i).File.FullName)
								QueuedFiles(i).IsProcessed = True
							Else
								QueuedFiles(i).File.Delete()
							End If
						End If
					Next
				End SyncLock
			End While
		Catch ex As Exception
			Console.WriteLine("Exception reported - {0}", ex)
		End Try
	End Sub
End Module

Class ProcessableFile
	Private ReadOnly fFile As FileInfo
	Public ReadOnly Property File() As FileInfo
		Get
			Return fFile
		End Get
	End Property

	Public Property IsProcessing() As Boolean
	Public Property IsProcessed() As Boolean

	Public Sub New(ByVal [file] As FileInfo)
		fFile = file
	End Sub
End Class

Open in new window

Which produces the following output -Capture.JPGAgain this is a very spartan example.  But should give you a good idea of where this could potentially head.

-saige-
smithmrkAuthor Commented:
Thank you all for your comments and suggestions and I will review them all and get back to you.  I'm on the road this week and probably won't get a chance to test any of these solutions out until next week.

Thanks again,
Mark
smithmrkAuthor Commented:
Perfect!!!
Thanks everyone!

Mark
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
ASP.NET

From novice to tech pro — start learning today.