[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 415
  • Last Modified:

Throttle FTP sending batch script

I have a large directory of 500,000 files.  I want to send to another computer via FTP which does some processing.  Because of the processing, I would like to throttle the speed of the send.  

Now, I do at CMD  
ftp> <ipaddres>
username,
password,
bin,
prompt,
mput.  

That will send the entire directory.  However, I would like it to send every file while waiting for 1 second inbetween files.  how can I throttle the send so it waits for one second in between files?

Thanks in advance.
0
weklica
Asked:
weklica
2 Solutions
 
Bill PrewCommented:
This should do what you want.  Change the folder as needed, and place the host, user and pswd values in where indicated...

@echo off
setlocal

REM Switch to the folder to xfer
pushd c:\temp

REM Process all files in folder, one at a time
for %%A in ("*.*") do (
  REM FTP file
  (
    echo open [hostname]
    echo [username]
    echo [password]
    echo bin
    echo put %File%
    echo quit
  ) | ftp -n -i

  REM Delay one second between files
  ping 1.1.1.1 -n 1 -w 1000>NUL
)

REM Return to original folder
popd

Open in new window

~bp
0
 
weklicaAuthor Commented:
If I leave in the brackets:

Unknown host [10.202.16.69].
Invalid command.
Invalid command.
Not connected.
Not connected.

Then, If take the brackets out:

Invalid command.
Invalid command.
Local file Remote file Invalid command.
0
 
Bill PrewCommented:
You don't want the brackets.

Try this, different FTP servers require slightly different connect sequences.

    echo open [hostname]
    echo user [username] [password]

~bp
0
Veeam Disaster Recovery in Microsoft Azure

Veeam PN for Microsoft Azure is a FREE solution designed to simplify and automate the setup of a DR site in Microsoft Azure using lightweight software-defined networking. It reduces the complexity of VPN deployments and is designed for businesses of ALL sizes.

 
weklicaAuthor Commented:
Hmmm.  Still not working.  However,
I have an FTP script that writes a little text file that I use to access.  This works well, but again, a whole directory at a time.... maybe this way could work.  I would prefer your way, but I don't want to be too bothersome.  Thanks!

echo %MYVAR42%>c:\Directory\DICOM_FILES\tempscriptfile1.txt
echo %MYVAR43%>c:\Directory\DICOM_FILES\tempscriptfile2.txt
echo bin>c:\Directory\DICOM_FILES\tempscriptfile3.txt
echo prompt>c:\Directory\DICOM_FILES\tempscriptfile4.txt
echo prompt>c:\Directory\DICOM_FILES\tempscriptfile5.txt
echo mput *.*>c:\Directory\DICOM_FILES\tempscriptfile6.txt
echo disconnect>c:\Directory\DICOM_FILES\tempscriptfile7.txt
echo bye>c:\Directory\DICOM_FILES\tempscriptfile8.txt
cd\
cd Directory\DICOM_FILES
copy /b /y tempscriptfile1.txt+tempscriptfile2.txt+tempscriptfile3.txt+tempscriptfi
le4.txt+tempscriptfile5.txt+tempscriptfile6.txt+tempscriptfile7.txt+temp
scriptfile8.txt DICOM_FILESscript.txt
ftp -i -s:DICOM_FILESscript.txt %MYVAR44%
0
 
weklicaAuthor Commented:
OK.  So, this works ALMOST:  the code below shows a version that works except it doesn't do the echo put %File% to the tempscriptfile6.txt file.  I know this is a train-wreck way to do it, but it is the only way I know how.  

So, is there a way to write put %File% to a text file since ECHO doesn't let %File% work after it?

@echo on
setlocal

REM Switch to the folder to xfer
pushd d:\Sandbox

REM Process all files in folder, one at a time
for %%A in ("*.*") do (
  REM FTP file
  (
del /q c:\tempscriptfile*txt
del d:\Sandbox\DICOM_FILESscript.txt
echo cd_importer>c:\tempscriptfile1.txt
echo cd_importer>c:\tempscriptfile2.txt
echo bin>c:\tempscriptfile3.txt
echo prompt>c:\tempscriptfile4.txt
echo prompt>c:\tempscriptfile5.txt
echo put %File%>c:\tempscriptfile6.txt
echo disconnect>c:\tempscriptfile7.txt
echo bye>c:\tempscriptfile8.txt
pause
c:
cd\
copy/b /y tempscriptfile1.txt+tempscriptfile2.txt+tempscriptfile3.txt+tempscriptfile4.txt+tempscriptfile5.txt+tempscriptfile6.txt+tempscriptfile7.txt+tempscriptfile8.txt DICOM_FILESscript.txt
copy c:\DICOM_FILESscript.txt d:\Sandbox
d:
cd\
cd Sandbox
ftp -i -s:DICOM_FILESscript.txt 10.202.16.69
  )

  REM Delay one second between files
  ping 1.1.1.1 -n 1 -w 1000>NUL
)

REM Return to original folder
popd

Open in new window

0
 
paultomasiCommented:
How about something like this:

@echo off

pushd d:\sandbox

for %%a in (*.*) do (
  call :makescript "%%a"
  ftp -i -s:"%temp%\script.txt" 10.202.16.69
  call :wait_1
)

del "%temp%\script.txt"
popd
exit /b


:makescript
  (
    echo cd_importer
    echo cd_importer
    echo bin
    echo prompt
    echo prompt
    echo put "%~1"
    echo disconnect
    echo bye
  )>"%temp%\script.txt"
goto :eof


:wait_1
  ping 1.1.1.1 -n 1 -w 1000>NUL
goto :eof

Open in new window

0
 
Bill PrewCommented:
I did notice one typo in my earlier code, this fixes that.  Give it a try and let me know what it displays.

@echo off
setlocal

REM Switch to the folder to xfer
pushd d:\Sandbox

REM Process all files in folder, one at a time
for %%A in ("*.*") do (
  REM FTP file
  (
    echo cd_importer
    echo cd_importer
    echo bin
    echo put %%A
    echo disconnect
    echo bye
  ) | ftp -n -i 10.202.16.69

  REM Delay one second between files
  ping 1.1.1.1 -n 1 -w 1000>NUL
)

REM Return to original folder
popd

Open in new window

~bp
0
 
Gerwin Jansen, EE MVETopic Advisor Commented:
Just a thought here: 500,000 sleeping seconds divided by 86,400 seconds in a day means that your script will run for around 6 days, excluding FTP/batch time needed. You can modify the "ping 1.1.1.1 -n 1 -w 1000>NUL" to 500 to sleep half a second....
0
 
weklicaAuthor Commented:
That wouldn't work either.  I will try other variations of it, but this is what does work:
@echo on
setlocal

REM Switch to the folder to xfer
pushd d:\Sandbox

REM Process all files in folder, one at a time
for %%A in ("*.*") do (
  (
del /q c:\tempscriptfile*txt
del d:\tempFTPqueue\DICOM_FILESscript.txt
echo cd_importer>c:\tempscriptfile1.txt
echo cd_importer>c:\tempscriptfile2.txt
echo bin>c:\tempscriptfile3.txt
echo prompt>c:\tempscriptfile4.txt
echo prompt>c:\tempscriptfile5.txt
echo put %%A>c:\tempscriptfile6.txt
echo disconnect>c:\tempscriptfile7.txt
echo bye>c:\tempscriptfile8.txt
echo %%A>>c:\ECHORestore.log
c:
cd\
copy/b /y tempscriptfile1.txt+tempscriptfile2.txt+tempscriptfile3.txt+tempscriptfile4.txt+tempscriptfile5.txt+tempscriptfile6.txt+tempscriptfile7.txt+tempscriptfile8.txt DICOM_FILESscript.txt
copy c:\DICOM_FILESscript.txt d:\Sandbox
d:
cd\
cd Sandbox
ftp -i -s:DICOM_FILESscript.txt 10.202.16.69
  )

  REM Delay one second between files
  ping 1.1.1.1 -n 1 -w 1000>NUL
)

REM Return to original folder
popd

Open in new window

0
 
Bill PrewCommented:
Interesting, since at this point they should be doing pretty much the same thing.

~bp
0
 
Bill PrewCommented:
What does "cd_importer" do?

And why do you do two "prompt" commands?  My understanding is that is a toggle command, so doing 2 is the same as doing none, right? Puts you back in whatever state you start in.

~bp
0
 
weklicaAuthor Commented:
cd_importer represents both the username and password to the FTP server.
0
 
weklicaAuthor Commented:
My solution is what I am using, but billprew's is more elegant and I always appreciate his help.
0
 
Bill PrewCommented:
Thanks!

~bp
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now