Solved

How do I estimate the bandwidth requirements of a conceptual website?

Posted on 2014-03-18
8
243 Views
Last Modified: 2014-03-28
I'm supposed to come up with cost estimates ... at 100k users ... at 1 million users.  I've estimated the hardware ... also the Amazon EC2 instances if we go that route.

But how do I estimate the bandwidth?  I've tried measuring bandwidth I use in a given time using a "comparable" website (using Chrome's developer tools).  My extrapolations have an "average" user using 480 MB/month from the regular server and 1.4 GB/month from the CDN.  Cost calculations at that level are crazy high.

What is a good estimation method?  Or do some of the big players publish their bandwidth usage?
0
Comment
Question by:Daniel Wilson
8 Comments
 
LVL 58

Expert Comment

by:Gary
Comment Utility
Your figures seem crazy for one user but...
What are they downloading? Are you accounting for cached content?
0
 
LVL 25

Expert Comment

by:Fred Marshall
Comment Utility
Have you considered cached content in the context of an edge provider like Akamai?
They might even be able to help you answer your question - or at least tell you that it's too small for them or .....
0
 
LVL 32

Author Comment

by:Daniel Wilson
Comment Utility
My "similar" site is Facebook.  I'm accumulating the MB's that Chrome's Developer Tools say are downloaded ... which appears to be measuring what's actually downloaded, not what's cached on the browser.

But I agree my figures seem crazy for a single user.  I really suspect my methodology is SERIOUSLY flawed.

How SHOULD I be estimating the usage?
0
 
LVL 58

Expert Comment

by:Gary
Comment Utility
Initial load count everything, subsequent loads count only the html page size.
You will have to guestimate the number of pages they will visit and what if any new content is on those pages e.g. images.

For example if the initial load is 500kb including all css, js, common images etc then subsequent pages are likely going to be around 10kb (or less) as all that is being downloaded is the HTML.  If you said some of those pages had a few extra images then just add a rough figure to cover those images to the base html size.
And remember once the HTML is downloaded it won't be downloaded again unless it is changed.
If you do have constantly changing pages then think about implementing ajax to update that content.
(Based on a 30+ day cache)
Even comparing to Facebook where you might have real time communication you are only talking about bits and bytes.
If they visited every day for a month and visited the same 5 pages you are likely only talking a few Mb's a month.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 32

Author Comment

by:Daniel Wilson
Comment Utility
Gary, I have the strong feeling you're right.

Can you suggest how to verify or quantify that?
0
 
LVL 58

Accepted Solution

by:
Gary earned 300 total points
Comment Utility
0
 
LVL 32

Assisted Solution

by:shalomc
shalomc earned 200 total points
Comment Utility
A small methodological correction:
It is wrong to assume that once an image is loaded is will always be served from browser cache unless changed.
Subsequent loads of a page may or may not load all or part of the page, images, css etc.
It depends on the caching policy of the object, on the expiry date, on the etag value, and on the client.
Create a top estimate and plan your caching policy accordingly, but be prepared for the top value.

For download visitor bandwidth: number of page views X average page size. Include estimate of images, cs, js, swf etc. Also add the total number of ajax traffic.

For bot traffic: add 100%. Bots are a major part of the "invisible" traffic coming to your web site. This includes google, msn, baidu and yahoo, as well as dozens of others. Also your health checks, your competition, content scrapers and others.

For upload traffic: appx 0.25K per hit. A hit is every request coming your way. EC2 don't charge for it yest, but good to know just in case.

Hardware/servers: Increasing traffic will force you place more servers whether you like it or not. A CDN can help a lot, but cannot alleviate this issue completely, especially if part of the content is dynamic (like from a CMS). Unfortunately there is no formula for this, I can only wish you so much traffic that you need to use a load balancer with a web server cluster.
0
 
LVL 32

Author Closing Comment

by:Daniel Wilson
Comment Utility
Thanks!
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Want to add social media icons to your email signature but not sure how to do it? Use this guide and learn how to add them quickly and easily.
Find out how to use dynamic social media in email signatures with this top 10 DOs & DON’Ts.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
The goal of the tutorial is to teach the user how to make an account for Skype and brief over view of all the options. There are three parts in this series.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now