How to make sure, file is copied completely ?

I have 2 apps: 1 is writing to a certain folder and another is reading from it and moves files to some other folders. Right now, second does not start until 1-st finishes. Would that be possible to run them concurrently ? The main concern is what if 2-nd app gets a hold of a file that is not copied completely ??

Any recommendations would be greatly appreciated

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

David ToddSenior DBACommented:

The basic problem is some form of communication between the two programs, so that 1st can let second knwo that there is another file to copy.

My first thought is why two programs? Why not simply create a function that writes to both locatiosn at the same time - eliminating the need for the copy?

I'd be more likley to combine this into one program, which controls both the original write and the copying.

three options.

one, have the two programs in one, as david sez.

two, have the 2nd prgram called by the first, so that only after copying is it called.

three, have some status file, say "Status.txt" deleted at teh beginning of copying byt the first program and created when the copying is done.
Hence, 2nd program wud look for existence of this file before copying.
in your app2 check if the files which are being copied are existing or not.

If they are existing then the files are copied or else they are either not copied or the copy process for the file is going on.

to check if the file is existing or not you can use the below condition

if (Dir(Str_Filename, vbNormal Or vbReadOnly Or vbHidden Or vbSystem Or vbArchive) <> "") = true then
   ''' file is existing
   '' file is not existing
end if


Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Mike TomlinsonMiddle School Assistant TeacherCommented:
You can't simply check to see if the file exists because the sytem creates a 0k file and then writes to it.  

I came across this type of problem when I wrote a complex piece of software that delivered and received files via another ftp app I had written.  The solution I came up with was to embed the file size in the file name itself.  Then if an app wants to know if the file is done being sent/received/copied etc, it can compare the expected file size from the file name, to the actual file size on the system.  If the two don't match then the file is still being worked on.  Once you are sure no more operations are being performed on the file (because the file name size matches the acutal size), you can remove the file size from the file name and do with it what you want.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Mike TomlinsonMiddle School Assistant TeacherCommented:
You could also simply rename files to be copied with a temporary extension and then copy/move the file and rename it back afterward.  Your second app just needs to ignore any file with your temporary extension because it knows it is being worked on.

There is a simple solution here.  When you create a new file you create it on the same drive but in a temp folder.  When the generation of the file is complete you move the file to the pickup folder.  The file is not recopied, the folder is just udated.  In this way the 2nd app can't see the file until it is ready for collection.

Name TempFile As DestinationFile

Furthermore, there are some bugs in VB's Dir command which can manifest when you delete files withing a Dir loop.  You must have on error active whenever you use Dir().  To avoid the problem pickup all files before you start to process them.  If you don't do this your app may work for a few months but it will keep screwing up.

How to cause the problem:

sFile = Dir(sFolder + "*.*")
Do While Len(SFile)>0
       ProcessFile sFolder, sFile ' do whaterver you need
       sFile = Dir() ' Get next file ** warning this can fail but only once in a blue moon

How to avoide the problem:

Dim Attempts as Long
dim colFiles As Collection
Dim vFile as Variant


If Attempts>5 Then
      msgbox "You have a problem"
      Exit Sub
End If

Attempts = Attempts +1
On Error GoTo Restart

Set colFiles = New Collection

sFile = Dir(sFolder + "*.*")
Do While Len(SFile)>0
    colFiles.Add sFile
    sFile = Dir()

On Error Goto xxxxx
' now files can be moved/deleted
For Each vFile in colFiles
       ProcessFile sFolder, vFile ' do whaterver you need

Furthermore,  there is a registry setting which causes an exefile to be run whenever a new file is loaded into a folder.  This is used by the Windows 2000 SMTP maildrop folder.  If you have SMTP loaded onto your server your can find the details of the setting be searching registry for your maildrop folder.
AlexF777Author Commented:

                    As surprising as it may seem, it takes so long to rename the file when sending stuff across the pipe, that I would rather parse out the file's name to get the size than do the renaming thing.

 Thank You Everyone.
Perhaps I'm missing something with how you need this to work but here's another suggestion.  If you're using built in file routines (as opposed to the File System Object) to write the file, you can set the access level:

Dim intOutFileNo as integer

intOutFileNo = FreeFile()

Open "C:\MyFolder\MyFile.txt" For Output Lock Read As #intOutFileNo
'Write your data
Close #intOutFileNo
In you file reading code, simply try to open the file.  If it's still being written to, you'll get an error 55 - File Already Open because of the read lock.  Just have your read code trap the error and try again later.  I don't have a network to test this on but it works locally.  Unfortunately the FSO doesn't include the lock syntax but I wouldn't be surprised if you still get an error if the file hasn't been closed by the writing application.  
AlexF777Author Commented:
Very interesting!

          I really do not need to read any of the files processed ( just move them around depending on their
names ), however, I guess, I could try to use your technique just to make sure, file is available.

         Thank You,
Exactly.  The open attempt is just to check the status, even though you don't really need to read the contents. It does have the advantage of not requiring a second copy operation or messing with the file names.   I don't know of another way to do this.  Even the api function OF_EXIST
"Opens the file and then closes it. Used to test for a file’s existence."   You could also set up a message queu between the two applications and have the first send a message notifying the second as each file was written and available for moving.  In any case, I'd try to keep the two as independent as possible so that changing one doesn't necesaarily break the other.
>  As surprising as it may seem, it takes so long to rename the file when sending stuff across the pipe, that I
 >  would rather parse out the file's name to get the size than do the renaming thing.

If the temporary file is saved on the save hard drive a file rename is instant.

In the my 'buttonpucher" VB-code for automatic printing postscript files from QuarkXPress I made comparsion between the present and past file sizes.

How it works. I need separately to print (write) QuarkXpress pages in the separate files. I cannot control inside the Quark becouse my app is just "buttonpusher" (use SendKeys statement). Therefore I should control for file existing and its size becouse it is not allowed to start print next page 2 (3, 4, 5, etc) until previous page is not done. I make this:

    For cw = 1 To 30 ' timeout
        fex = False ' fex - indicator
        myTimer 1  ' timeout 1 second
            If fso.FileExists(sf) Then
                Set fil = fso.GetFile(sf) ' if file created then object activated
                fex = True
                Exit For
            End If
    Next cw
    If fex = False Then MsgBox "Page " & sf & " was not printed to file!", vbCritical, "Error": Exit Sub

    pfs = 0 ' pfs - previous file size, long
    For cw = 1 To 60
        fex = False
        myTimer 2
        If fil.Size = pfs Then
            fex = True ' if in during 2 seconds file size not changed this means that write process is done
            Exit For
        End If
        pfs = fil.Size
    Next cw
    If fex = False Then MsgBox "Page " & sf & " was not printed to file!", vbCritical, "Error": Exit Sub

Private Function myTimer(myPause As Double) ' floating variable
Dim Start As Double
   Start = Timer
   Do While Timer < Start + myPause
        If endStop Then Exit Function
End Function

End of

Everything is works fine.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic Classic

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.