Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Where do big websites store all their uploads?

Posted on 2012-03-17
7
376 Views
Last Modified: 2012-03-18
Probably kind of a newb question here, but I guess I don't really know the answer.

Sites like YouTube, Facebook, which probably have tens of thousands of uploads to them per day... do they just continually add hard drives to increase storage space for all this? Seems kind of impractical.  I remember back in the day, you were only allowed to upload a certain amount of data per user account. How do they handle the mass amounts of data constantly being uploaded to those sites? It must be thousands of gigs if not terabytes per day. Where's it all being stored?
0
Comment
Question by:Tymetwister
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 3

Accepted Solution

by:
IMIronMan earned 170 total points
ID: 37733423
If I tried to explain it....you wouldn't believe me, so I let you take a look a how Google does it.  It's AMAZING!

http://www.youtube.com/watch?v=zRwPSFpLX8I
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 37733457
That's a cool video.  Here's more info: http://www.google.com/about/datacenters/  If you click on the 'Locations' link, you'll see that Google has at least 10 locations for data centers.  Microsoft, Yahoo, Facebook, Apple, Godaddy and other major internet sites have similar setups.  And there are data centers for people you never heard of.  One data center in Colorado has 1,000,000 sq ft of floor space and room for 262,000 servers and of course, all that support equipment to run them.
0
 
LVL 8

Author Comment

by:Tymetwister
ID: 37733537
I guess I'm still not absorbing the heart of what I was asking about. So... it pretty much is just adding an unthinkable amount of HDD's and storage space to house all of it in datacenters?
0
Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 130 total points
ID: 37733640
Yep.  That Google video said it held 45,000 servers and each one would be a new machine in I think they said 2009.  That probably means 500GB or larger drives in each server.  And that is only 1 of 10 locations that they have around the world.  So some quick arithmetic comes to at least 450,000 hard drives with at least 22,000 Terabytes... and that is Just Google (Youtube is part of Google now).  Granted, it takes a while to install all those servers so you could say it just keeps on growing.  I wonder how long their servers last and how often they replace them with new machines.

You can Google info on the data centers for most of the large organizations.
0
 
LVL 33

Expert Comment

by:shalomc
ID: 37734767
And don't forget that some of theses datacenters are used for cloud based storage - they rent out storage to anyone who wants it. Check out Amazon, Google, Microsoft Azure, RackSpace, Gogrid, Xerox, and others....

That's how many projects who need tons of storage start, when the risk does not justify building their own datacenters.
0
 
LVL 3

Assisted Solution

by:IMIronMan
IMIronMan earned 170 total points
ID: 37734885
And we keep getting bigger drive technology....Check out this news story:

A data repository almost 10 times bigger than any made before is being built by researchers at IBM's Almaden, California, research lab. The 120 petabyte "drive"—that's 120 million gigabytes—is made up of 200,000 conventional hard disk drives working together. The giant data container is expected to store around one trillion files and should provide the space needed to allow more powerful simulations of complex systems, like those used to model weather and climate.

A 120 petabyte drive could hold 24 billion typical five-megabyte MP3 files or comfortably swallow 60 copies of the biggest backup of the Web, the 150 billion pages that make up the Internet Archive's WayBack Machine.
0
 
LVL 8

Author Closing Comment

by:Tymetwister
ID: 37735318
All very interesting facts. I feel like I have a better understanding of how it works now. Thanks all.
0

Featured Post

ScreenConnect 6.0 Free Trial

Explore all the enhancements in one game-changing release, ScreenConnect 6.0, based on partner feedback. New features include a redesigned UI, app configurations and chat acknowledgement to improve customer engagement!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Each year, investment in cloud platforms grows more than 20% (https://www.immun.io/hubfs/Immunio_2016/Content/Marketing/Cloud-Security-Report-2016.pdf?submissionGuid=a8d80a00-6fee-4b85-81db-a4e28f681762) as an increasing number of companies begin to…
The question appears often enough, how do I transfer my data from my old server to the new server while preserving file shares, share permissions, and NTFS permisions.  Here are my tips for handling such a transfer.
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question