Batch File : Sync FTP to local folder

hi Renege,
u can find many ftp sync utility out there, syncing FTP through batch would be a tedious task i guess if the files are inside more than one directory on ftp server.

On the ftp server, the files are just in root folder or in different sub dirs ?

You won't get very far with robocopy, because it doesn't know anything about ftp.
Try wget, I've used it some years ago for a similar purpose.
GNU Wget
http://www.gnu.org/software/wget/

Obda, I guess he wants to use  Batch only..as usual may be another experiment  by ReneGe

@subhashchy:
-You are almost right about the experiment part. However, I need this to be done quickly.
-Yes, it does have subfolders, so I guess It may get messy...

@oBdA:
-I used robocopy in my example so you get the idea.
-wget seems to be a good solution for this.

Would either on of you mind providing me with the script by using wget?  Onless you have a better idea.

Thanks again,
Rene

I think your best shot if you need a pure BAT solution would be to FTP down the entire contents of the remote FTP folder with MGET to a work folder locally, and then use robocopy from that work folder to the folder to be synced.

But there really are better tools out there for this task , if you want to go that way let me know and I'll pass along a couple of suggestions.

~bp

If you want a tool to get a solution quickly how about the free version here:

http://www.cyberkiko.com/page/FTPSync.aspx

~bp

Wouldn't that just the the LS command?  But I think I've seen the format of the output can be hard to parse afterwards.

~bp

Actually sorry, it's the DIR subcommand you want, not LS.

~bp

yeah. i tried LS as well but didn't sound good to me , gonna be at least 20 trip to ftp and back to local cmd line if try to make it using LS.

there are only two sub commands for dir inside FTP, -a and -l , both not gonna help :(

@kdyer:
Thanks for your hard work, but I am lost in VBScript, so I can't use it.

@subhashchy:
Thanks for scripting this for me.

@billprew:
Thanks for the app, I had it a quick try and did not work. I guess I need more time to figure it out.
If you know it, would you mind sharing me an example of a ini file with the required command line so I can sync files from the FTP site to a local folder?

Cheers,
Rene

A simple

wget.exe -m ftp://username:password@domainname.com:1111

Select allOpen in new window

should do the trick.
User name and password could be passed as --ftp-user and --ftp-password as well.

@AlexPace:
Thanks for your script. However, RoboFTP is not free, but rather very expensive,  so I can't use it

@oBdA:
I don't know how I can do the required sync with your command line. Sorry for that ;-(

Cheers,
Rene

==> Thanks for the app, I had it a quick try and did not work. I guess I need more time to figure it out.
==> If you know it, would you mind sharing me an example of a ini file with the required command line ==> so I can sync files from the FTP site to a local folder?

What files do you want to sync from the ftp host?  Is it all the files in the default folder when you connect via FTP, or is it a particular named subfolder from there? Do you want all files or just certain ones matching a patern?

~bp

@billprew:
All files, folders, sub-folders...

Thanks bp

Okay, here's the batch commands:

@echo off
ftpsync "EE27287357" /FTPSYNCDATA:"C:\EE\TEMP" /QUIET
if errorlevel 1 echo Errors in FTP sync

Select allOpen in new window

and here's the INI file (C:\EE\TEMP\EE27287357.ini):

[Source]
Server=ftp.server.com
User=username
Pass=password
Dir=/
Type=U
Case=Sensitive
IncludeSubDirs=Yes

[Destination]
Type=F
Dir=C:\EE\TEMP\Files\
Case=Sensitive

Select allOpen in new window

~bp

Another program that will do want you want and is much more reasonably priced is Beyond Compare.

http://www.scootersoftware.com/moreinfo.php?zz=pricing

It can run in batch mode and is also very handy for a lot of other things as well.

@vk3kjc:
Thanks for your efforts and code. Considering that I am looking for a free solution and this software is way to greeddy... So I can't use it.

@bp:
Trying it now!

Thanks and cheers

FWIW, I was able to do something useful with WGET as well, following the example from oBdA earlier.

~bp

@bp

I did not work for me (Errors in FTP sync).

See below:

Thanks

@echo off

REM CREATING INI FILE
	SET IniFile=%~dpn0.ini
	IF EXIST "%IniFile%" DEL "%IniFile%"
	
	SET ini.1=[Source]
	SET ini.2=Server=ftpsite.com:2222
	SET ini.3=User=username
	SET ini.4=Pass=password
	SET ini.5=Dir=/eetest
	SET ini.6=Type=U
	SET ini.7=Case=Sensitive
	SET ini.8=IncludeSubDirs=Yes
	SET ini.9=[Destination]
	SET ini.a=Type=F
	SET ini.b=Dir=C:\temp\ee
	SET ini.c=Case=Sensitive

	FOR /F "tokens=1* delims==" %%A in ('SET ini.') DO ECHO %%B>>"%IniFile%"

	REM CALL :RealIniValues


ftpsync "%IniFile%" /FTPSYNCDATA:"C:\temp\ee" /QUIET
if errorlevel 1 echo Errors in FTP sync
pause

Select allOpen in new window

On the ftpsync command line, the INI file name parm does not allow the INI extension, it is assumed.  Notice my example line of:

ftpsync "EE27287357" /FTPSYNCDATA:"C:\EE\TEMP" /QUIET

which read C:\EE\TEMP\EE27287357.ini

~bp

Maybe something like this:

@echo off
REM CREATING INI FILE
SET IniName=%~dpn0
SET IniFile=%IniName%.ini
(
ECHO [Source]
ECHO Server=ftpsite.com:2222
ECHO User=username
ECHO Pass=password
ECHO Dir=/eetest
ECHO Type=U
ECHO Case=Sensitive
ECHO IncludeSubDirs=Yes
ECHO [Destination]
ECHO Type=F
ECHO Dir=C:\temp\ee
ECHO Case=Sensitive
)>"%IniFile%"
ftpsync "%IniName%" /FTPSYNCDATA:"C:\temp\ee" /QUIET
if errorlevel 1 echo Errors in FTP sync
pause

Select allOpen in new window

~bp

Hold on, that's not going to work either, since you have the path in the ININAME, just a sec...

~bp

This should be closer.  The INI file needs to be in the folder specified by /FTPSYNCDATA.

@echo off
REM CREATING INI FILE
SET IniName=%~n0
SET IniFile=C:\temp\ee\%~n0.ini
(
ECHO [Source]
ECHO Server=ftpsite.com:2222
ECHO User=username
ECHO Pass=password
ECHO Dir=/eetest
ECHO Type=U
ECHO Case=Sensitive
ECHO IncludeSubDirs=Yes
ECHO [Destination]
ECHO Type=F
ECHO Dir=C:\temp\ee
ECHO Case=Sensitive
)>"%IniFile%"
ftpsync "%IniName%" /FTPSYNCDATA:"C:\temp\ee" /QUIET
if errorlevel 1 echo Errors in FTP sync
pause

Select allOpen in new window

~bp

@bp:

Since I am not using the default FTP port 21, port parameter may not be entered correctly.

http://www.cyberkiko.com/docs/ftpsync28/pPort.htm?MenuState=X3BAAAAAAEEAAAAAQAFUVAAAAUA

However, they do not give instructions as in the ini file, like for example "port=2222".

Have any new ideas?

Thanks

Here is the /DEBUG log file content

I changed the domain name for privacy.

And by the way, by using the same parameters, I can access the FTP site with FTP client Filezilla.

Cheers

CyberKiko FTPSync 2.8f
Copyright © 1996-2011 by Kristof Gajsek, http://www.cyberkiko.com. All Rights Reserved.
This product includes software developed by Francois Piette & TeamICS (http://www.overbyte.be/)

Priority: Normal
Data folder: C:\temp\ee\
Loading MTB file: C:\temp\ee\sync.mtb

Start: 02-sept.-2011 14:47:05
  Syncing C:\Temp\ee\rene\ with ftpsite.com:2222/
  
  Detecting FTP mode for ftpsite.com:2222, please wait (this may take up to 10 minutes)...
  Once FTPSync determines FTP mode it will save it for all subsequent transfers.
  WARNING: FTPSync can not determine FTP mode. Passive mode assumed. Please set manually!
  Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /

Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /

Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /

Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /

Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /

Too many connection failures. Last error: 500 DNS lookup error - No Data (#11004) (CONNECT) on /


  
  Source FTP unreachable: 500 DNS lookup error - No Data (#11004)
  Source FTP unreachable: 500 DNS lookup error - No Data (#11004)
  Source FTP unreachable: 500 DNS lookup error - No Data (#11004)
  Source FTP unreachable: 500 DNS lookup error - No Data (#11004)

Select allOpen in new window

Looks to me like you would replace:

ECHO Server=ftpsite.com:2222

with:

ECHO Server=ftpsite.com
ECHO Port=2222

~bp

@bp:

It works... youhoooo...

1 last detail (please):

The auto detect, detects "passive mode".
FTP mode: Passive
Debug: > PASV
Debug: < 227 Entering Passive Mode

So I added "Passive=Yes". But still starts auto-detecting again and takes an eternity.

Have any ideas?

Passive=Yes in the [source] section seems like it would be the right approach. When I did a little testing here it didn't take long at all to autodetect, how long is this "eternity" you mention ;-).

~bp

2 minutes 5 seconds

Thanks BP, you nailed it !!

Would you mind sharing what you found about WGET that would sync FTP site to local folder?

Here is my final E.E. working version.

Thanks a million and cheers!

@echo off

SET Retry=10
SET WorkingFolder=%~dp0WorkingFolder
SET IniFile=%WorkingFolder%\%~n0.ini
SET IniName=%~n0
SET IniTempFile=%~dpn0.ini
SET FTPLocalSyncFolder=C:\FTPLocalSyncFolder

IF NOT EXIST "%WorkingFolder%" MD "%WorkingFolder%"
IF NOT EXIST "%FTPLocalSyncFolder%" MD "%FTPLocalSyncFolder%"

REM IF NOT EXIST, CREATING INI FILE
	IF NOT EXIST "%IniFile%" (
		ECHO FIRST TIME RUNNING. MAY TAKE FEW MINUTES TO START
		SET /P User=ENTER FTP USER NAME: 
		SET /P Pass=ENTER FTP PASSWORD : 
	)
	
	IF NOT EXIST "%IniFile%" (
		ECHO [Source]
		ECHO Server=ftpserver.com
		ECHO Port=22
		ECHO User=%User%
		ECHO Pass=%Pass%
		ECHO Dir=/
		ECHO Type=U
		ECHO Case=Sensitive
		ECHO IncludeSubDirs=Yes
		ECHO Passive=Yes

		ECHO [Destination]
		ECHO Type=F
		ECHO Dir=%FTPLocalSyncFolder%
		ECHO Case=Sensitive
	)>"%IniFile%"

CLS
ECHO STARTING FTP SYNC

:Repeat
IF %Retry% EQU 0 GOTO End
SET /a Retry-=1
ftpsync "%IniName%" /DEBUG /FTPSYNCDATA:"%WorkingFolder%" /QUIET
if errorlevel 4 ECHO ERROR NUMBER [%errorlevel%] RETRY NUMBER [%Retry% OF 10] & goto Inierror
if errorlevel 3 ECHO ERROR NUMBER [%errorlevel%] RETRY NUMBER [%Retry% OF 10] & goto Repeat
if errorlevel 2 ECHO ERROR NUMBER [%errorlevel%] RETRY NUMBER [%Retry% OF 10] & goto Repeat
if errorlevel 1 ECHO ERROR NUMBER [%errorlevel%] RETRY NUMBER [%Retry% OF 10] & goto Repeat
GOTO end

:Inierror
ECHO "ERROR: Most likely there's a problem in ["%IniFile%"] file!"

:End
EXIT

Select allOpen in new window

I just CD to the folder I want the files in, and then did as mentioned above:

wget.exe -m ftp://username:password@domainname.com:1111

~bp

@ bp and oBdA

With:
wget.exe -m --passive -P LocalFolder ftp://username:password@domainname.com:1111

I was able to mirror from the FTP site. However, when I delete a file at the FTP site, it is not deleted at the local folder.  This is why I was unable to use WGET. Unless I did something wrong?

Have any ideas about this?

Thanks

You have done ennuf on this thread. I will close it and open a new one for WGET.

https://www.experts-exchange.com/questions/27289892/Batch-File-WGET-to-fully-mirror-FTP-site.html

I will split points according to contribution and effort.

Thanks and cheers,
Rene

@oBdA

Sorry for not responding to your message. I just didn'nt see it until bp quoted your solution:
https://www.experts-exchange.com/questions/27287357/Batch-File-Sync-FTP-to-local-folder.html?cid=1575&anchorAnswerId=36473643#a36473643

Going to bed now!  Zzzzzz

Thanks and cheers,
Rene