Advice on a batch file that copies, moves, sftp's & deletes files

I wrote 3 batch jobs to run in order at 15, 30 and 45 minutes past each hour to process some orders which works fine but I have been asked for the whole process to run every 15 minutes so combined all 3 of these into one batch file as below.  My only changes are to move the "exit" command from the end of the SFTP part of the script and added the @echo off sleep 120 command between each script so that the previous parts of the script are allowed 2 minutes to process before the next part of the script starts.

While I am waiting for the supplier to give me to go ahead to send test files I wondered if someone could comment as to whether the script was ok or something else added to improve it

The one issue I have is the files are picked up immediately at the destination server and processed so I am unable to use a script to only delete once files have been cross checked on the destination server.

Copy /y "D:\GenesisIntegration\XXX\PO\*.*" "D:\GenesisIntegration\XXX\XXXFiles\"
Copy /y "D:\GenesisIntegration\XXX\RETURNS\*.*" "D:\GenesisIntegration\XX\XXXFiles\"
Copy /y "D:\GenesisIntegration\XXX\GRN\*.*" "D:\GenesisIntegration\XXX\XXXFiles\"
Move /y "D:\GenesisIntegration\XXX\PO\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"
Move /y "D:\GenesisIntegration\XXX\RETURNS\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"
Move /y "D:\GenesisIntegration\XXX\GRN\*.*" "D:\GenesisIntegration\XXX\XXXArchive\"

@echo off
sleep 120

"C:\Program Files (x86)\WinSCP\" /command "open s" "put D:\GenesisIntegration\XXX\XXXFiles\* /genesis/" 

@echo off
sleep 120

Del /Q "D:\GenesisIntegration\XXX\XXXFiles\*.*"


Open in new window

Matthew EmeryHomeAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

The one issue I have is the files are picked up immediately at the destination server and processed so I am unable to use a script to only delete once files have been cross checked on the destination server.

you need to handle errors returned by your sftp command and only delete the files if the command was successful
Bill PrewIT / Software Engineering ConsultantCommented:
In general your script looks okay.  I typically try to use variables rather than hard coding paths and file names through out the script, so wanted to show how that approach works.  Makes it easier to read going forward, and much easier to maintain if you need to move or change the folders involved.  Also used a small look to clean up the copy and move code, again more for educational benefit that because yours wouldn't work.

Always try and add some comments along the way in your scripts, you and others will be glad you did in the future and it's good practice.  I added a few but feel free to add more.

No need for the exit at the end of the script, when the script reaches its end it stops executing.  Or was that intended to be a command that you need to send to WinSCP at the end?

I moved the @echo off lines to a single one at the top, typically that's what you want and it stays set for the entire script unless changed with an @echo on later.

I don't think there is an easy way for you to know when the files have actually been acted on or processed at the receiving end before you remove them, so don't think you can accomplish that.

Hope this is useful.

@echo off

rem Define paths and folders to work with
set BaseDir=D:\GenesisIntegration\XXX
set FilesDir=D:\GenesisIntegration\XXX\XXXFiles
set ArchiveDir=D:\GenesisIntegration\XXX\XXXArchive
set FtpDir=/genesis
set CopyFolders=PO,RETURNS,GRN
set WinScp=C:\Program Files (x86)\WinSCP\

rem copy and archive new files for each subfolder
for %%A in (%CopyFolders%) do (
  copy /y "%BaseDir%\%%A\*.*" "%FilesDir%\"
  move /y "%BaseDir%\%%A\*.*" "%ArchiveDir%\"

rem Pause 2 minutes
sleep 120

rem FTP new files to server
"%WinSCP%" /command "open s" "put %FilesDir%\* %FtpDir%/"

rem Pause 2 minutes
sleep 120

rem Delete new files
del /q "%FilesDir%\*.*"

Open in new window


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Matthew EmeryHomeAuthor Commented:
Thanks for both comments:

I need to find a better way of handling issues with Ftp'd files, maybe the supplier needs to delay their processing task so I can setup a file check.

With regards to what you have provided Bill that is very helpful.

1) I was initially after a command to take only the contents of 3 folders and FTP them but as I did not know how to copy the files only I added the first part of the script to move them all to the consolidated directory first.

2) I added the exit command to the WinSCP command initially as the scheduled tasks continued to run without the exit command being there.

Very happy with what you have done, it is better for others to understand and makes good sense!
Build an E-Commerce Site with Angular 5

Learn how to build an E-Commerce site with Angular 5, a JavaScript framework used by developers to build web, desktop, and mobile applications.

you probably can simplify further

for %%A in (%CopyFolders%/*) do (
  if "%WinSCP%" /command "open s" "put %BaseDir%\%%A %FtpDir%/" (
    move /y "%BaseDir%\%%A\*.*" "%ArchiveDir%\"

Open in new window

note that this connects using scp once for each file which is not ideal either.
i'm not used to winscp but the regular sftp would allow scripting this within the sftp command so the connection stays open. maybe that's feasible in winscp, and that would be much better.
Matthew EmeryHomeAuthor Commented:
Thank you so much for the help.

This is working a treat I do not need build in 1 last command check to ensure remote server connection is open first which is first on my list for Monday
Matthew EmeryHomeAuthor Commented:
Almost forgot

Instead of the sleep command I have used

Ping -n -w 10000>nul

And a variation of this as it seems to work better
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.