?
Solved

How do we know when a file is finished uploading?

Posted on 2008-11-13
16
Medium Priority
?
257 Views
Last Modified: 2013-11-10
The client will upload 2 files at any time during the day. We have an SSIS package than runs every 15 minutes. The first step within the SSIS package calls a .bat file to determine if the files are present. If the file is present the SSIS package will continue processing.

What we want to do is to determine that not only that the files exist, but that the upload is complete. Is there any way MS DOS can determine this, or can something be done in the SSIS package to determine that the upload is complete?
0
Comment
Question by:thirdwavellc
  • 6
  • 5
  • 3
  • +2
16 Comments
 
LVL 17

Expert Comment

by:HoggZilla
ID: 22953555
The files are uploaded to ? A database table?
In SSIS you can create a number of indicators to let you know the process is complete. You could even rename the file using a dos command through the execute process task in Control Flow. What do you want?
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22953566
Sure, use the send mail task.  Then drag the on completion line (precedence constraint) over to it.
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22953602
the above is in SSIS
0
Veeam Disaster Recovery in Microsoft Azure

Veeam PN for Microsoft Azure is a FREE solution designed to simplify and automate the setup of a DR site in Microsoft Azure using lightweight software-defined networking. It reduces the complexity of VPN deployments and is designed for businesses of ALL sizes.

 

Author Comment

by:thirdwavellc
ID: 22953669
Simply put, the client is uploading two files.

Can a DOS batch script determine if these files have finished uploading?
OR
Can a task in SSIS determine if these files have finished uploading?

The files are uploaded to a folder on the server, not a database.
0
 
LVL 17

Expert Comment

by:HoggZilla
ID: 22953704
You need to determine if the client is still uploading the file, not in the middle and you grab them before they are complete.
Can the client wirte an indicator file that you look for to know they are done?
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22953708
Yeah that's fine, the last step in the SSIS package should be the FTP process.  In visual studio on the left choose the send mail task and put in all your information.  After that drag the green line from ftp task to the send mail task.  That way whenever the ftp completes it will pump out an e-mail to whoever you tell it to.
0
 

Author Comment

by:thirdwavellc
ID: 22953753
Gallitin,

We are not FTPing the file in the SSIS package. The client is FTPing the files to a folder on the server, then we run the SSIS package.

But we only want the SSIS package to run if the files have finished uploading.

0
 

Author Comment

by:thirdwavellc
ID: 22954575
HoggZilla,

I have asked our client to see if he could upload an additional text file after the other two files have been uploaded as an indicator. This is a good idea, although it does not directly answer the question posted.

I'll reply to this posting with the answer from the client when I hear back from him.
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22954633
thirdwavellc, sorry I didn't see that before.  Why not have the SSIS package download the file then download the file again and if the file sizes are not = to each other then loop it to download again otherwise it's finished.
0
 

Author Comment

by:thirdwavellc
ID: 22954981
Gallitin,

Sorry, that's not what I'm looking for. If the file is 100+ MB, this will eat up to much processing power.
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22955035
In your current SSIS package why not call for the file size then delay 5-10 seconds then call for it again and compare the two values.  If they = each other then continue and download otherwise stop.  That way you wouldn't have to download the files.
0
 

Author Comment

by:thirdwavellc
ID: 22955158
Gallitin,

This is the kind of solution I'm looking for. Can this happen in a DOS command? If so do you have any code? OR can this be done in SSIS and if so do you have any samples I can see?
0
 
LVL 12

Expert Comment

by:Nathan Riley
ID: 22955416
I would assume it could be done in DOS, but why would you want to when you already have a SSIS package.  I'm assuming your downloading the files thru a Script Task in that package?  If so I would just add the logic in there.
0
 
LVL 30

Expert Comment

by:nmcdermaid
ID: 22956406
I think the indicator file would be the most reliable. Checking the size is OK but what happens if the upload is interrupted for say 30 seconds, then continues. You might mistake that its complete when its not.
Assuming the client is FTPing then maybe there is something you can hook into on the FTP server side (your side) to indicate that the file is complete. i.e maybe the FTP server has an expected file size or a checksum.
This page:
http://en.wikipedia.org/wiki/File_Transfer_Protocol
says
No integrity check on the receiver side. If a transfer is interrupted, the receiver has no way to know if the received file is complete or not
Which implies that no, there is no way to check when a file is complete. But its worth reading your FTP server documentation.
0
 
LVL 29

Accepted Solution

by:
Bill Bach earned 800 total points
ID: 22960232
If the file is still uploading, a DOS command RENAME will fail to rename the file.

As such, you can probably make SSIS call the REN command and simply check the return code.  If 0, the rename was successful.  If you gate back a 1, then it failed.  Alternatively, you could create a batch file to sleep pending the file's completion (with a small "sleep" loop), but I think an upload of a large file might have an impact on your process.  

To check it in a batch file (your original question), use this:
   REN filename  newfilename
   IF %ERRORLEVEL% == 0 ...

Using pure DOS, you're limited in what you can do to GOTO statements and the like, but I suspect that you're on a Win32 platform, so you can use the more powerful scripting function to put other stuff in place of the "..."
0
 
LVL 17

Expert Comment

by:HoggZilla
ID: 22967253
That DOS RENAME is a great answer. Way to go BillBach!!! At least it is something in DOS.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A Stored Procedure in Microsoft SQL Server is a powerful feature that it can be used to execute the Data Manipulation Language (DML) or Data Definition Language (DDL). Depending on business requirements, a single Stored Procedure can return differe…
Ready to get certified? Check out some courses that help you prepare for third-party exams.
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.
Via a live example, show how to shrink a transaction log file down to a reasonable size.
Suggested Courses

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question