Advice on a batch file that copies, moves, sftp's & deletes files

Matthew Emery
Matthew Emery used Ask the Experts™
on
I wrote 3 batch jobs to run in order at 15, 30 and 45 minutes past each hour to process some orders which works fine but I have been asked for the whole process to run every 15 minutes so combined all 3 of these into one batch file as below.  My only changes are to move the "exit" command from the end of the SFTP part of the script and added the @echo off sleep 120 command between each script so that the previous parts of the script are allowed 2 minutes to process before the next part of the script starts.

While I am waiting for the supplier to give me to go ahead to send test files I wondered if someone could comment as to whether the script was ok or something else added to improve it

The one issue I have is the files are picked up immediately at the destination server and processed so I am unable to use a script to only delete once files have been cross checked on the destination server.

Copy /y "D:\GenesisIntegration\XXX\PO\*.*" "D:\GenesisIntegration\XXX\XXXFiles\"
Copy /y "D:\GenesisIntegration\XXX\RETURNS\*.*" "D:\GenesisIntegration\XX\XXXFiles\"
Copy /y "D:\GenesisIntegration\XXX\GRN\*.*" "D:\GenesisIntegration\XXX\XXXFiles\"
Move /y "D:\GenesisIntegration\XXX\PO\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"
Move /y "D:\GenesisIntegration\XXX\RETURNS\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"
Move /y "D:\GenesisIntegration\XXX\GRN\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"

@echo off
sleep 120

"C:\Program Files (x86)\WinSCP\WinSCP.com" /command "open sftp://xxxxxxxxxxx:xxxxxxxxxxxxxx@xxx.xxx.xxx.xxx/" "put D:\GenesisIntegration\XXX\XXXFiles\* /genesis/" 

@echo off
sleep 120

Del /Q "D:\GenesisIntegration\XXX\XXXFiles\*.*"

"exit"

Open in new window

Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
The one issue I have is the files are picked up immediately at the destination server and processed so I am unable to use a script to only delete once files have been cross checked on the destination server.

you need to handle errors returned by your sftp command and only delete the files if the command was successful
IT / Software Engineering Consultant
Top Expert 2016
Commented:
In general your script looks okay.  I typically try to use variables rather than hard coding paths and file names through out the script, so wanted to show how that approach works.  Makes it easier to read going forward, and much easier to maintain if you need to move or change the folders involved.  Also used a small look to clean up the copy and move code, again more for educational benefit that because yours wouldn't work.

Always try and add some comments along the way in your scripts, you and others will be glad you did in the future and it's good practice.  I added a few but feel free to add more.

No need for the exit at the end of the script, when the script reaches its end it stops executing.  Or was that intended to be a command that you need to send to WinSCP at the end?

I moved the @echo off lines to a single one at the top, typically that's what you want and it stays set for the entire script unless changed with an @echo on later.

I don't think there is an easy way for you to know when the files have actually been acted on or processed at the receiving end before you remove them, so don't think you can accomplish that.

Hope this is useful.

@echo off
setlocal

rem Define paths and folders to work with
set BaseDir=D:\GenesisIntegration\XXX
set FilesDir=D:\GenesisIntegration\XXX\XXXFiles
set ArchiveDir=D:\GenesisIntegration\XXX\XXXArchive
set FtpDir=/genesis
set CopyFolders=PO,RETURNS,GRN
set WinScp=C:\Program Files (x86)\WinSCP\WinSCP.com

rem copy and archive new files for each subfolder
for %%A in (%CopyFolders%) do (
  copy /y "%BaseDir%\%%A\*.*" "%FilesDir%\"
  move /y "%BaseDir%\%%A\*.*" "%ArchiveDir%\"
)

rem Pause 2 minutes
sleep 120

rem FTP new files to server
"%WinSCP%" /command "open sftp://xxxxxxxxxxx:xxxxxxxxxxxxxx@xxx.xxx.xxx.xxx/" "put %FilesDir%\* %FtpDir%/"

rem Pause 2 minutes
sleep 120

rem Delete new files
del /q "%FilesDir%\*.*"

Open in new window

~bp

Author

Commented:
Thanks for both comments:

I need to find a better way of handling issues with Ftp'd files, maybe the supplier needs to delay their processing task so I can setup a file check.

With regards to what you have provided Bill that is very helpful.

1) I was initially after a command to take only the contents of 3 folders and FTP them but as I did not know how to copy the files only I added the first part of the script to move them all to the consolidated directory first.

2) I added the exit command to the WinSCP command initially as the scheduled tasks continued to run without the exit command being there.

Very happy with what you have done, it is better for others to understand and makes good sense!
Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

you probably can simplify further

for %%A in (%CopyFolders%/*) do (
  if "%WinSCP%" /command "open sftp://xxxxxxxxxxx:xxxxxxxxxxxxxx@xxx.xxx.xxx.xxx/" "put %BaseDir%\%%A %FtpDir%/" (
    move /y "%BaseDir%\%%A\*.*" "%ArchiveDir%\"
  )
)

Open in new window


note that this connects using scp once for each file which is not ideal either.
i'm not used to winscp but the regular sftp would allow scripting this within the sftp command so the connection stays open. maybe that's feasible in winscp, and that would be much better.

Author

Commented:
Thank you so much for the help.

This is working a treat I do not need build in 1 last command check to ensure remote server connection is open first which is first on my list for Monday

Author

Commented:
Almost forgot

Instead of the sleep command I have used

Ping 1.1.1.1 -n -w 10000>nul

And a variation of this as it seems to work better

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial