?
Solved

How to move 200 Gigs to Amazon S3

Posted on 2016-09-08
4
Medium Priority
?
139 Views
Last Modified: 2016-09-09
What is the best way to transfer 200 gigs of 100kb images from a Dedicated Server to Amazon S3.
0
Comment
Question by:cxs410
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 58

Expert Comment

by:Julian Hansen
ID: 41790466
Amazon Snowball is a possible option you can read more about it here
http://docs.aws.amazon.com/AWSImportExport/latest/ug/whatissnowball.html
AWS Import/Export is a service that accelerates transferring large amounts of data into and out of AWS using physical storage appliances, bypassing the Internet. AWS Import/Export has two features that support this functionality in two different ways, AWS Snowball (Snowball) and AWS Import/Export Disk (Disk).
0
 

Author Comment

by:cxs410
ID: 41790546
I was hoping for some kind of server to service connection.
0
 
LVL 5

Accepted Solution

by:
Laroy Shtotland earned 2000 total points
ID: 41790564
Server to service connection? If you are talking about AWS Direct Connect  it's not the case for it.

Data Transfer IN To Amazon S3 is free. Your 2097152 PUT requests (200 GB = 209715200 KB = 209715200 KB / 100 KB) will cost you 10$, if you need each of the files as an object, so it doesn't matter how you upload those files.

You will not be able to use multipart upload APIs for multiple threads in parallel for objects less than 15MB,

I think a faster way is to fire up a free EC2 micro instance, upload zip archives there, unzip, and then PUT those files to S3 from EC2.

I also heard about www.cloudzipinc.com but I haven't had an opportunity to try it myself yet.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 41791054
Of course your connection speed to the Internet/AWS is going to factor in how long its going to take to upload,

200GBytes on a 100Mbit link is probably going to take in excess of 10 Hours
0

Featured Post

Learn how to optimize MySQL for your business need

With the increasing importance of apps & networks in both business & personal interconnections, perfor. has become one of the key metrics of successful communication. This ebook is a hands-on business-case-driven guide to understanding MySQL query parameter tuning & database perf

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When crafting your “Why Us” page, there are a plethora of pitfalls to avoid. Follow these five tips, and you’ll be well on your way to creating an effective page.
In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller singl…
The viewer will get a basic understanding of what section 508 compliance can entail, learn about skip navigation links, alt text, transcripts, and font size controls.
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question