Solved

How to move 200 Gigs to Amazon S3

Posted on 2016-09-08
4
86 Views
Last Modified: 2016-09-09
What is the best way to transfer 200 gigs of 100kb images from a Dedicated Server to Amazon S3.
0
Comment
Question by:cxs410
4 Comments
 
LVL 54

Expert Comment

by:Julian Hansen
ID: 41790466
Amazon Snowball is a possible option you can read more about it here
http://docs.aws.amazon.com/AWSImportExport/latest/ug/whatissnowball.html
AWS Import/Export is a service that accelerates transferring large amounts of data into and out of AWS using physical storage appliances, bypassing the Internet. AWS Import/Export has two features that support this functionality in two different ways, AWS Snowball (Snowball) and AWS Import/Export Disk (Disk).
0
 

Author Comment

by:cxs410
ID: 41790546
I was hoping for some kind of server to service connection.
0
 
LVL 5

Accepted Solution

by:
Laroy Shtotland earned 500 total points
ID: 41790564
Server to service connection? If you are talking about AWS Direct Connect  it's not the case for it.

Data Transfer IN To Amazon S3 is free. Your 2097152 PUT requests (200 GB = 209715200 KB = 209715200 KB / 100 KB) will cost you 10$, if you need each of the files as an object, so it doesn't matter how you upload those files.

You will not be able to use multipart upload APIs for multiple threads in parallel for objects less than 15MB,

I think a faster way is to fire up a free EC2 micro instance, upload zip archives there, unzip, and then PUT those files to S3 from EC2.

I also heard about www.cloudzipinc.com but I haven't had an opportunity to try it myself yet.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 41791054
Of course your connection speed to the Internet/AWS is going to factor in how long its going to take to upload,

200GBytes on a 100Mbit link is probably going to take in excess of 10 Hours
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Developer portfolios can be a bit of an enigma—how do you present yourself to employers without burying them in lines of code?  A modern portfolio is more than just work samples, it’s also a statement of how you work.
Because your company can’t afford for you to make SEO mistakes, you’ll want to ensure you’re taking the right steps each and every time you post a new piece of content. This list of optimization do’s and don’ts can help you become an SEO wizard.
The viewer will learn how to count occurrences of each item in an array.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

785 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question