[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now


Transferring large amounts of small files by FTP: slow, excessive bandwidth

Posted on 2008-11-20
Medium Priority
Last Modified: 2013-11-29
I recently had to upload approximately 1,500 JPEG image files to my website. The total upload size was around 17Mb.

I used Windows built-in FTP transfer and the Internet connection is ADSL2+.

Firstly it took well over 12 hours to upload to complete which seems odd given that I frequently upload single large files in the same way and it is usually very quick. Is it because there were lots of small files that it took signifcantly longer?

The other strange thing about this was my Internet connection usage. I expected to see around 17Mb uploaded but my ISP reports that I DOWNLOADED (received) about 2000Mb and uploaded (sent) about 200Mb during this time. I typically use about 100Mb DL/20Mb UL a day so the majority of the other figures are all down to this FTP upload.

Is FTP a very inefficient method of transferring this type of data? Would it be better if I used another FTP client like Filezilla rather than the one in Windows? Are the above data transfer figures what is to be expected for this type of data transfer?
Question by:knobbylowboy

Accepted Solution

jansvetter earned 750 total points
ID: 23002310
1500 Files produce a whole lot of overhead, considering you used the ftp functionality of IE would explain some of the traffic, but not 2 Gigabytes. That seems to be kind of a connection problem, best guess is that the server you're uploading the data to dismisses it and so you're resending it a hundred times.

Best Tip would be to try FileZilla as a client (use the portable version, keeps your'e systems reg. clean), works pretty well for me.

=>  http://portableapps.com/apps/internet/filezilla_portable
LVL 15

Expert Comment

ID: 23002321
Do your server host support extracting zip files at server side?
 IF yes then you can just upload a single zip file and do the unzip after upload. It will save you a lot of time.

Author Comment

ID: 23010744
I downloaded and used Filezilla and did a smaller transfer of about 250 files, 5MB, and it was done in a matter of only a few minutes. Looks like Microsoft Windows Explorer FTP transfer is very slow and inefficient. I would be interested to hear if other people have found this too. (Or perhaps I am the only one foolish enough to use it for such a transfer.)

Featured Post

Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Understanding FTPS File transfer is a common requirement in most Enterprises. While there are numerous ways to get a file from Point A to Point B over a network, perhaps the most common method still in use is FTP – File Transfer Protocol. FTP is …
Configuring network clients can be a chore, especially if there are a large number of them or a lot of itinerant users.  DHCP dynamically manages this process, much to the relief of users and administrators alike!
Viewers will learn how to properly install and use Secure Shell (SSH) to work on projects or homework remotely. Download Secure Shell: Follow basic installation instructions: Open Secure Shell and use "Quick Connect" to enter credentials includi…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Suggested Courses
Course of the Month19 days, 22 hours left to enroll

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question