How to check if file has completed FTP

dataflowjoe
dataflowjoe used Ask the Experts™
on
Hi Guys,

I've written a program that processes and converts CSV files into XLS files. The problem I've got is that the files are FTP'd from one server into a folder on another server that I'm monitoring every 5 seconds, sometimes incomplete files are being processed because they haven't been fully FTP'd. How can I tell if a file has been fully FTP'd before it is allowed to be processed, is there a file attribute that I can use to tell me if it has been fully copied?

Thanks for your help here.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
file should be locked while FTPing, so try this:

   If Not FileLocked(csvFileName) Then
       'process into XML
   End If
Jim Dettman (EE MVE)President / Owner
Most Valuable Expert 2017
Most Valuable Expert 2012
Commented:
<< How can I tell if a file has been fully FTP'd before it is allowed to be processed, is there a file attribute that I can use to tell me if it has been fully copied?>>
Unfortunately no and this is the one big drawback with FTP; it doesn't lock files. What you need to do is also transmit a "flag file". ie.
MyFile.Dat is the file
and
MyFile.Dat.Sent
is the flag file. The flag file can have anything in it. You send that file after your done sending the first file. in that way, the receiving end knows that it can now process the first file.
JimD.
Top Expert 2009
Commented:
I don't know how much data you can get back from the FTP site in your code, but it might be possible to just compare file size. When the local file size equals the FTP file size, it's done.
Or you could use JDettman's suggestion, but add a "failsafe". In the extra file you send after the main file, put the file size in it. That way you'll know for sure the file made it to the FTP in it's entirety.
Should you be charging more for IT Services?

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Top Expert 2010
Commented:
Just upload the files with temporary extension like (myfile.temp) and then rename it to it's real extension. When you process the FTP entries known extension are considered complete.
The question is did you write the FTP side of this or is it another third part program.

Author

Commented:
Hi Guys,
Thank you all for your valuable comments which as usual pointed me in the right direction. I've awarded the points as per the inspiration received!
For those interested the solution I came up with, it's as follows.... It's been tested and seems to work!!

'put this code into separate bas module
'bas module Code start ===========

Private Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)
Private Declare Function GetFileSize Lib "kernel32" (ByVal hFile As Long, lpFileSizeHigh As Long) As Long


Public Sub PauseProcess(lngMilliSecs As Long)
On Error Resume Next
    Sleep (lngMilliSecs)
End Sub

Public Function GetFileS(FilePath As String) As Long
On Error GoTo errHandler
    Dim Pointer As Long, sizeofFile As Long
    Pointer = lOpen(FilePath, OF_READ)
    'size of the file
    sizeofFile = GetFileSize(Pointer, lpFSHigh)
    lclose Pointer
    GetFileS = sizeofFile
Exit Function
errHandler:
    GetFileS = 0
    Resume Next
End Function


'bas module Code end =============


'code in my processing form start ======
Sub ProcessFile(strFileName as string)

 Dim lngFileSize1 As Long
 Dim lngFileSize2 As Long
 Dim sngTimer As Single


        lngFileSize1 = GetFileS(strFileName)
        sngTimer = Timer + 60
       
        Do
            PauseProcess 1000
            lngFileSize2 = GetFileS(txtNetworkFolder & strFileName)
            If lngFileSize1 = lngFileSize2 Then Exit Do
            If Timer > sngTimer Then Exit Do
            lngFileSize1 = GetFileS(txtNetworkFolder & strFileName)
        Loop

      if lngFileSize1 = 0 then
            exit sub
            else
            'convert csv to excel starts here.....      
      end if

End sub

'code in my processing form end ======
Jim Dettman (EE MVE)President / Owner
Most Valuable Expert 2017
Most Valuable Expert 2012

Commented:
<< lngFileSize2 = GetFileS(txtNetworkFolder & strFileName)>>
 Not sure I understand; if you have access to the other server via a path, then why FTP at all?   You could simply copy the file.
JimD.

Author

Commented:
Hi Jim,

I don't have access to the remote server via a path. I'm checking the files as they get FTP'd in to the mapped folder I'm monitoring every 5 seconds. The FTP process happens randomly from a remote server in another country , I'm just polling the mapped folder where the files are being transferred to and that's where I was getting the problem with partial transference of the file. Checking the file size every second was what I needed to do.  When the file size becomes constant between the 1 second file size checks  then I know that the file has been fully transferred and the conversion process can begin.

Joe

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial