Solved

How to move 200 Gigs to Amazon S3

Posted on 2016-09-08
4
118 Views
Last Modified: 2016-09-09
What is the best way to transfer 200 gigs of 100kb images from a Dedicated Server to Amazon S3.
0
Comment
Question by:cxs410
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 57

Expert Comment

by:Julian Hansen
ID: 41790466
Amazon Snowball is a possible option you can read more about it here
http://docs.aws.amazon.com/AWSImportExport/latest/ug/whatissnowball.html
AWS Import/Export is a service that accelerates transferring large amounts of data into and out of AWS using physical storage appliances, bypassing the Internet. AWS Import/Export has two features that support this functionality in two different ways, AWS Snowball (Snowball) and AWS Import/Export Disk (Disk).
0
 

Author Comment

by:cxs410
ID: 41790546
I was hoping for some kind of server to service connection.
0
 
LVL 5

Accepted Solution

by:
Laroy Shtotland earned 500 total points
ID: 41790564
Server to service connection? If you are talking about AWS Direct Connect  it's not the case for it.

Data Transfer IN To Amazon S3 is free. Your 2097152 PUT requests (200 GB = 209715200 KB = 209715200 KB / 100 KB) will cost you 10$, if you need each of the files as an object, so it doesn't matter how you upload those files.

You will not be able to use multipart upload APIs for multiple threads in parallel for objects less than 15MB,

I think a faster way is to fire up a free EC2 micro instance, upload zip archives there, unzip, and then PUT those files to S3 from EC2.

I also heard about www.cloudzipinc.com but I haven't had an opportunity to try it myself yet.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 41791054
Of course your connection speed to the Internet/AWS is going to factor in how long its going to take to upload,

200GBytes on a 100Mbit link is probably going to take in excess of 10 Hours
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When speed and performance are vital to revenue, companies must have complete confidence in their cloud environment.
Australian government abolished Visa 457 earlier this April and this article describes how this decision might affect Australian IT scene and IT experts.
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question