• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 432
  • Last Modified:

How does Flickr and Facebook handle Pictures

Hello Experts,
 How does the big boys handle pictures of web?
1. Do they write the blob to a sql db?
2) Do the the build a folder for each user and store the pix on disk aas well as store an index to that directory/ picture in the sql db?

3) I am using a Hosting Company with a Virtual Server any suggestions of a larger outfit that may handle the large banwidth?  Or do you think that most hosting services are ok? I am anticiapating 40k users.

I am using C#,  ASP.net and SQL 2008.

I want to allow my users to add pictures. Any amount of pix but not larger than 10 mb.
is AJAX the way to go for File Uploads/Store?

I realize these questions are pedestrian, but I am working alone and have noone now to bounce these dumb questions off of.



2 Solutions
For both security and performance, you should consider a dedicated server with that many users, especially if a large percentage will be using it simultaneously.

I don't really know, but I'd be willing to bet the big guys use databases for images.  Many sites use AJAX for their upload setups because it looks fancier and can be customized more than a simple "browse" button will give you, such as thumbnails with checkboxes.  I notice the same sites let you opt out for a simpler upload interface.
1. To db is the only way it could possibly scale. File access is slow and inherently risky from a security standpoint, although they could dump the uploaded files onto the file system before streaming the binary data to the db. Much the way e-mails are dumped to an outbound directory by one process and actually sent out by another process. That would prevent the upload from bogging down under load, and allow anti-virus anti-malware to check the files before dumping them in a db.

2. I'd be looking for a dedicated server with the thought of mirroring the photo data across many servers dispersed geographically. This is how larger companies handle large repositories of binary data which will be accessed by a large audience. In fact, there are media hosting operations which just handle the data for other web sites.  Inktomi for example acts as a a proxy cache for web traffic and on-demand streaming media.  (Images, Ads, and audio.) Yahoo bought them and they are used by many hosts to serve media to users from the closest server.

3. At far less than 40K users, a financial services site I developed (read project lead with a team of programmers) also had the web server mirrored at three locations in the US, New York, Texas, and California.  From my point of view it was one site to maintiain, but the company paid a pretty penny for the hosting provider to keep the dedicated servers mirrored and running.

The right web hosting provider (notice how I avoid a direct recommendation) will be happy to provide a copy of their SAS 70 certification and at least some clients of similar size as your proposed operation for you to contact for reference. They won't give you their entire client portfolio, but should have some clients you can contact.  They will also negotiate their prices for a sizable web operation, but may require an up front long term capital commitment to negotiate the best prices.

Sounds like more than a one person development project to me.  

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now