Link to home
Start Free TrialLog in
Avatar of swcrook
swcrookFlag for United States of America

asked on

WinSCP - How to confirm file download before deleting remote file

Hi experts.

I would like to know if there is a way to confirm a file has been successfully downloaded before deleting it on a remote server. I have a very basic batch script (running from console command/scheduled task)that goes out to a remote SSH server and downloads 2 files each night. Currently, I have everything working successfully, but I would like to add a "checks and balances" option to make sure that I have a successful download of the files before WinSCP deletes them via the "rm" command. Also, one of the files that I download isn't always there because it doesn't nee to be generated on the remote side (*-*-.reject.txt), so if it isn't there is there a way to tell WinSCP to ignore that command or is it ok to just leave it as it? Everything seems to work great except I would like to check the files download before deleting.

I know there is probably a better way to do this, so that is why I ask the experts. Here is what I use:

# Automatically answer all prompts negatively not to stall
# the script on errors
option batch on
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect using a password
# open user:password@example.com
# Connect
open xxxxxx@xxxxx.xxxxxxxx.com
# Change remote directory
cd /home/user
# Force binary mode transfer
option transfer binary
# Download file to the local directory d:\
get deer*.txt d:\
get *-*-reject.txt d:\
rm deer*.txt
rm *-*-reject.txt
# Disconnect
close
exit

Thanks all!
Avatar of Josh Miller
Josh Miller
Flag of United States of America image

You will have to split your winscp script into two scripts.  Download your files, check them and then run another winscp script to do the deleting.  One way to check to see if it downloaded completely is to download the file twice, with different names (or into two different folders) and then compare them.  You can use DOS's FC command to compare two files.

FC file1.txt file2.txt
if %errorlevel%==0 (
  REM files match ok, run winscp script to delete from server
) else (
  REM files don't match, don't delete from sever
)

Open in new window

Avatar of swcrook

ASKER

Ok, I believe I follow what you logic is but I am curious about a few things.

1) If I have two scripts, then the first will download the files twice with one file going to one folder and the same file going to a second folder or I can just rename it on the download the second time. With this scenario, and a completed download, how can I code it so that that second file that was used to confirm the completed download ( via FC in DOS ) is deleted. I obviously only need one copy. I havea programmer who processes this file and then archives it. I do not want a second copy of the file sitting in limbo.

2) If and only if the completed download completes I would want to delete both the downloaded second file and the file from the remote server.

I am an infrastructure guy, not a programmer, so please understand I am able to keep up with you most of the time but created code from scratch is something I have to research. If I have asked the questions in a silly fashion, then that is my only defense :)

Thanks for the help.
Your question makes sense, just add an extra "DEL" line to the if statement (shown below).  As for how to delete the file from the server on successful download, you have to make a second winscp script that skips the 'GET' lines and just does the 'RM' lines as you had at the bottom of the script you posted in your original question.  Also be sure to remove the 'RM' lines from your first winscp script that is only doing the file downloading now, not the removing.

rm deer*.txt
rm *-*-reject.txt
# Disconnect
close
exit
FC file1.txt file2.txt
if %errorlevel%==0 (
  del file2.txt
  REM files match ok, run winscp script to delete from server
) else (
  REM files don't match, don't delete from sever
)

Open in new window

Avatar of swcrook

ASKER

I will give it a shot tonight. Thanks for the help.
Avatar of swcrook

ASKER

Actually, here is where I am confused.

I believe you only took into account downloading just one file; whereas, we will possibly have two file to deal with. We will always have one file to download no matter what, in this example "deer*.txt". The second file will only needed to be downloaded if it is actually present. So, here is what I can see needs to happen, but I could be way off base here.

1) Run first script which downloads the files twice. I can successfully do that now and have it renaming on the second download. Here is the script for that:

=========================================
# Automatically answer all prompts negatively not to stall the script on errors
option batch on
# Disable overwrite confirmations that conflict with the previous
option confirm off
# Connect
open gasftp@datatxfr.xpresschex.com:26
# Change remote directory
cd /home/user
# Force binary mode transfer
option transfer binary
# Download file to the local directory d:\
get deer*.txt d:\
get *-*-reject.txt d:\
get deer*.txt d:\fileDeer.txt
get *-*-reject.txt d:\fileReject.txt
# Disconnect
close
exit
==========================================

My confusion now sets in because with the FC DOS code you have me it us only dealing with on .txt file and comparing that file, so my assumption is that I simply just need to run the set of code twice: once for the "deer*.txt" and once for the "*-.-.reject.txt". So, I would have something like this:

# Confirms if download was successful, if so deletes file downloaded twice
FC d:\deer*.txt d:\fileDeer.txt
if %errorlevel%==0 (
  del fileDeer.txt
  REM files match ok, run winscp script to delete from server
) else (
  REM files don't match, don't delete from sever
)

# Confirms if download was successful, if so deletes file downloaded twice
FC d:\*-*-reject.txt d:\fileReject.txt
if %errorlevel%==0 (
  del fileReject.txt
  REM files match ok, run winscp script to delete from server
) else (
  REM files don't match, don't delete from sever
)

However, when I run this in the first script it doesn't work. I am not sure with the winscp.exe command line understands FC, but I could be wrong. So, where do I place the "FC code" within the script or does that need to be a second script with the deletion script being the third script. Also, I am thinking that doubling the code like I have it probably isn't the most efficient, but I am not a programmer so I don't know. It appears like the code you provided could somehow call the next script (deletion script) if the files actually compare.

Can you give me what I am doing wrong in the scenario as I have explained it? Thanks again.
First, you can't have a file in DOS/Windows with an asterisk * in the filename.  What name does the file have when it is downloaded?  Is it just -.-.Reject.txt without the asterisks?  You can't have asterisks in filenames or on the FC command line.  It helps to know the exact name of the files you'll be dealing with.  If the filenames change, we will need to code some more logic into the batch file.

Second, sorry, I guess I didn't specify where this code goes.  The code I've been giving you is Windows Batch file ("DOS" / CMD ) code.  It goes into a batch file with a .CMD or .BAT extension (it doesn't matter which).  This batch file will call your WINSCP script where I have the REM lines (REM is short for remark - a comment).  The batch file code (the FC command, if statement, etc) doesn't go in the winscp script, the winscp script gets called FROM the batch file.  I'll illustrate what else you need to do here...

To test for both files, I'm going to change the logic a bit as well...
@ECHO OFF
REM Put this code into a text file with a .BAT or .CMD extension
 
START /WAIT WINSCP /console /script=DOWNLOADscriptfilename
 
FC d:\deer.txt d:\fileDeer.txt >nul
if %errorlevel%==1 GOTO :EOF(
  REM files don't match, don't delete from sever
) else (
  REM first file matches, now test second...
  FC d:\--reject.txt d:\fileReject.txt >nul
  if %errorlevel%==0 (
    REM files match ok, run winscp script to delete from server
    del d:\fileDeer.txt & del d:\fileReject.txt
    WINSCP /console /script=DELETEscriptfilename
  )
)

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of Josh Miller
Josh Miller
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of swcrook

ASKER

Thank you for your help. I have an acceptable solution that will work for me. I will give you the points because you helped me come to the solution I used, though it isn't exactly as you have stated. Thanks again for your time.
Avatar of swcrook

ASKER

Thanks for help.