Powershell Script Help

Hello,

I have the following powershell script which executes a command when a new file is created / uploaded / moved to a specified location. The command executes a SSIS package which needs the entire file contents to be present before running. My issue is, I do not want it to execute the command until the move / upload is completely finished. Sometimes large files get uploaded (5gb) and if the powershell script executes the command before the entire file move / upload is complete, the SSIS package will fail.

How can I modify this powershell script to wait until the entire file has been uploaded / moved?

$folder = 'W:\Uploads'
$filter = '*.*'                             # <-- set this according to your requirements
$cmd = 'dtexec.exe /FILE W:SSIS\Import.dtsx'
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property @{
 IncludeSubdirectories = $false             # <-- set this according to your requirements
 NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}
$onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
 invoke-expression -command $cmd
}

Open in new window



Thanks!
brandonc7Asked:
Who is Participating?
 
namethisCommented:
You assume that the file will always get uploaded completely at some time. But this may not be the case. The file may be uploaded partially even if another application stopped writing to it.
Or it may be damaged. You need to check file integrity in some way.
One way is to get the file archived before upload, and check archive integrity after upload.
Another way is to get md5- or sha hash along with the file.
0
 
brandonc7Author Commented:
Modified original question
0
 
brandonc7Author Commented:
Maybe a better way to say this is to make sure the file is not 'locked' by another application?
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
arnoldCommented:
One way is to look for a change file that was modified a certain time before, I.e. look at the modify date on the file to be five minutes ago this way you will be certain that no process is still writing to it.
You do not want to act on created.
0
 
brandonc7Author Commented:
Hmmmmm good point. The uploads are being done via FTP so I am limited because up to client as far as what they send. My ssis script is smart enough to catch a damaged file I think. That being said, is it possible to just modify this power shell script to detect lock / unlock status?
0
 
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
AFAIK FTP does not lock a file while transferring.
That is the reason we usually take an approach of copying the file to a different file name or folder, and after finishing let the FTP script perform a local rename/move to the target name. Another approach is to create a "lock file" before starting the transfer, and deleting it when finished. Of course that requires that you have control over the FTP script, which seems not to be an option here.

Sadly, FSW does not support a scanning interval, delayed triggering or triggering on no change for some time ... So the only reliable way IMO is to have a scheduled task (or looping script) checking for files older than e.g. 2 seconds. Or wait in the FSW event for at least 1 second, and recheck if the LastWrite date has changed.
0
 
AlexPaceCommented:
You could use a scriptable FTP server that can run a script when an upload is complete.  Here is a Robo-FTP Server event script that automatically moves uploaded files to another folder: http://kb.robo-ftp.com/script_library/show/46 

If you had something like that, your powershell script could monitor the destination folder instead of the FTP upload folder.... or you could just make a Robo-FTP Server event script launch the powershell script directly and pass in the name of the uploaded file.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.