• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 4159
  • Last Modified:

How can I transfer large files over the internet with resume capabilities?

The end goal is the ability to transfer large files (5-10gb) over the internet between two computers. The ability to pause the transfer would be a nice feature to have but isn't a requirement. The resume feature is a requirement, especially while uploading. I was thinking an FTP server might be the right choice but I'm not able to confirm which software has the ability to pause and resume file transfers. I did test with Windows Server 2008 R2 (IIS 7.5) as the FTP server and WinSCP as the client. While uploading a file from the client to the server I would break the connection and then reconnect it. WinSCP acknowledges the loss connection and restarts the upload when the connection is live again. The problem is; it starts over from the beginning instead of resuming where it left off. To me resume means; pick up where you left off, not start over.

It doesnt have to be FTP, Im open to try anything if it will accomplish what is needed. Please be specific on the server and client software needed.

0
jswan
Asked:
jswan
  • 9
  • 8
  • 2
  • +1
4 Solutions
 
CSecurityCommented:
Install a good FTP server like FileZilla or Microsoft's IIS FTP server, etc.

As client use FileZilla's own client or any other professional client, like CuteFTP, IPSwitch FTP.

Then you'll have pause/resume/continue feature.
0
 
jswanAuthor Commented:
I'm already using Microsoft's IIS FTP server in my testing, it's the one included with Server 2008 R2. I tried FileZilla before trying WinSCP and it wouldn't resume either, I see an option to resume ACSII files but I'm tranfering binary files. CutePDF did not work either and IPswitch is the only one that will pause, when you hit play it want to overwite the file so I don;t see the point to that.
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
TolomirAdministratorCommented:
Take a look at http://www.cerberusftp.com/

File Transfer Integrity Checking
SOAP control API
Active Directory Authentication with FTP Security Groups
LDAP and LDAPS Authentication
Resumes failed file transfers
Windows Service Support
IPv6 Support
0
 
jswanAuthor Commented:
CSecurity; I tested ALFTP and resume only works on downloads, on uploads it restarts from the beginning like the others.

Tolomir; have you used Cerberus FTP Server? What FTP client did you use? Did it do what I'm asking for?

The point of me asking the experts is to get real solutions that you have experience with, not just marketing material from websites. I appreciate the fast responses! Keep them coming!
0
 
TolomirAdministratorCommented:
0
 
mottakuttyCommented:

The best free tool for you is robocopy
Robocopy will resume file transfer and has a default retry period of 30 seconds.

Mottai
0
 
jswanAuthor Commented:
mottakutty; does Robocoy work over the internet (WAN)? I assume it will need some port open on the firewall??? Can you be more specific on the use of it?
0
 
TolomirAdministratorCommented:
As far as I know, robocopy is for networkshares at max.
In the tread above, I've mentioned they suggest the free filezilla.
0
 
jswanAuthor Commented:
Tolomir; In the thread it mentions it works for downloads but not uploads. I'm looking for resume on uploads... maybe I'm asking for the impossible. Do you have any experience with Cerberus FTP? I think I'll send their support an email to see what they say.
0
 
TolomirAdministratorCommented:
0
 
jswanAuthor Commented:
I assumes the FTP portion of IIS 7.5 fully supports resume on uploads too, can you confirm if it does or does not? I'm willing to setup Cerberus FTP if I can confirm it works... already have Windows Server 2008 R2 with IIS 7.5 up and running.
0
 
TolomirAdministratorCommented:
when the siteadmin of the cerberus ftp  forum says "yes it works", I personally would trust it.

 
0
 
mottakuttyCommented:


Robocopy is most suitable for file transfers over the WAN. This is because of its bulit in retry on disconnection feature.  If you are able to ping the two computers and access the drives by unc robocopy will do the job.

Cheers
Mottakutty
0
 
TolomirAdministratorCommented:
Ok wan is not Internet.
Also the need for encryption during transfer is not needed, since data will be at no time accessible by unauthorized externals.
Please tell us more about your setup. If you can use cifs/smb there is no need to install an ftp service.
Tolomir
0
 
jswanAuthor Commented:
I don't have UNC access; the transfer will be done over the internet so FTP does seem appropriate.


I sent this to Cerberus tech;

Before I purchase your software I have some technical questions about a feature (Resumes failed file transfers). I want the ability to resume files that have broken while uploading. Example; FTP client is used to transfer a 4GB media file (binary) to a Cerberus FTP server, during the upload the connection is lost, when the connection is restored I want the upload to continue automatically from the place it was when the connection was lost. If the file was 75% uploaded it should only need to upload the other 25%. Is this possible? Will it work as I explained? What version of your product do I need to use? What FTP client do you recommend?


Here is their response;

Yes, Cerberus FTP Server fully supports server resumes and resuming a failed file transfer is possible as you explained it.  You can use nearly any popular FTP client (though I would not recommend a web browser-based FTP client). You can use Filezilla FTP client to easily resume failed file transfers.  SmartFTP and WS_FTP client are also popular solutions.


Testing Cerberus FTP;

I installed Cerberus FTP and it looks like it will work! I've tested with Filezilla, SmartFTP and WS_FTP. Not sure which client I like best, leaning towards SmartFTP but then again Filezilla is free. I can connect and resume uploads with FTP and FTP/SSL (AUTH SSL), at this point I'm unable to connect using FTPS/Implicit SSL. Not sure why... seems to be a problem with a configuration within Cerberus FTP as I disabled the firewall on the server and made it a DMZ on the router / firewall and it still wouldn't work. Any suggestions on setting up FTPS/Implicit SSL with Cerberus FTP?
0
 
TolomirAdministratorCommented:
I found this:

I dunno the technicals on this, but I dug around for the "normal" FTPS ports and found that FTPS needs port 990 opened up for TCP and 989 for UDP. I dunno if it needs UDP or not so I opened um both up and presto.

http://www.cerberusftp.com/phpBB3/viewtopic.php?f=6&t=1439&p=6620&hilit=ssl#p6620
0
 
jswanAuthor Commented:
This what I've settled on;
Cerberus FTP server
Filezilla for FTP client
FTP over Explicit TSL/SSL
Firewall Settings;
Forward Ports (FTP 21), (FTPS 989 & 990) and (FTP Passive 11000 - 13000) all are TCP

I can pause and resume uploads by toggling the process queue (CTRL-P) in Filezilla and if the connection is lost between the client and server, uploads will resume automatically once it's restored.

More from Cerberus;

We've long since deprecated and removed Implicit SSL from Cerberus FTP Server.  Implicit SSL was a stopgap measure that has long since been discouraged for normal FTP use.  We've thought about adding it back in but we haven't seen much interest.  Explicit SSL is currently the only method of establishing an SSL connection.  We are also in the process of adding SFTP but it will be a few more months before that is ready.


More information;

What is the difference between Implicit SSL and Explicit SSL?
http://www.smartftp.com/support/kb/what-is-the-difference-between-implicit-ssl-and-explicit-ssl-f189.html

I believe that Cerberus only supports explicit SSL and not implicit SSL if that helps any.
http://www.cerberusftp.com/phpBB3/viewtopic.php?f=2&t=1553

Definitions;
FTP = File Tranfer Protocol
SFTP = SSH FTP
FTPS = FTP over Implicit TSL/SSL
FTPES  (AUTH SSL) = FTP over Explicit TSL/SSL

Thanks for all your help!
0
 
jswanAuthor Commented:
Had to do too much research on my own.
0
 
jswanAuthor Commented:
FYI - Cerberus was very responsive in answering my questions and that was also consider when chossing Cerberus FTP.
0
 
TolomirAdministratorCommented:
I know Cerberus is one of this great old tools, still active though.

Support either by forum or by email is something one cannot appreciate enough.

Great that you found your solution and also thank you for the details you provided from their support, for the users here..

Tolomir
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

  • 9
  • 8
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now