Solved

How do I estimate the bandwidth requirements of a conceptual website?

Posted on 2014-03-18
8
255 Views
Last Modified: 2014-03-28
I'm supposed to come up with cost estimates ... at 100k users ... at 1 million users.  I've estimated the hardware ... also the Amazon EC2 instances if we go that route.

But how do I estimate the bandwidth?  I've tried measuring bandwidth I use in a given time using a "comparable" website (using Chrome's developer tools).  My extrapolations have an "average" user using 480 MB/month from the regular server and 1.4 GB/month from the CDN.  Cost calculations at that level are crazy high.

What is a good estimation method?  Or do some of the big players publish their bandwidth usage?
0
Comment
Question by:Daniel Wilson
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 58

Expert Comment

by:Gary
ID: 39937245
Your figures seem crazy for one user but...
What are they downloading? Are you accounting for cached content?
0
 
LVL 26

Expert Comment

by:Fred Marshall
ID: 39937340
Have you considered cached content in the context of an edge provider like Akamai?
They might even be able to help you answer your question - or at least tell you that it's too small for them or .....
0
 
LVL 32

Author Comment

by:Daniel Wilson
ID: 39937582
My "similar" site is Facebook.  I'm accumulating the MB's that Chrome's Developer Tools say are downloaded ... which appears to be measuring what's actually downloaded, not what's cached on the browser.

But I agree my figures seem crazy for a single user.  I really suspect my methodology is SERIOUSLY flawed.

How SHOULD I be estimating the usage?
0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 58

Expert Comment

by:Gary
ID: 39937609
Initial load count everything, subsequent loads count only the html page size.
You will have to guestimate the number of pages they will visit and what if any new content is on those pages e.g. images.

For example if the initial load is 500kb including all css, js, common images etc then subsequent pages are likely going to be around 10kb (or less) as all that is being downloaded is the HTML.  If you said some of those pages had a few extra images then just add a rough figure to cover those images to the base html size.
And remember once the HTML is downloaded it won't be downloaded again unless it is changed.
If you do have constantly changing pages then think about implementing ajax to update that content.
(Based on a 30+ day cache)
Even comparing to Facebook where you might have real time communication you are only talking about bits and bytes.
If they visited every day for a month and visited the same 5 pages you are likely only talking a few Mb's a month.
0
 
LVL 32

Author Comment

by:Daniel Wilson
ID: 39937867
Gary, I have the strong feeling you're right.

Can you suggest how to verify or quantify that?
0
 
LVL 58

Accepted Solution

by:
Gary earned 300 total points
ID: 39937877
0
 
LVL 33

Assisted Solution

by:shalomc
shalomc earned 200 total points
ID: 39942771
A small methodological correction:
It is wrong to assume that once an image is loaded is will always be served from browser cache unless changed.
Subsequent loads of a page may or may not load all or part of the page, images, css etc.
It depends on the caching policy of the object, on the expiry date, on the etag value, and on the client.
Create a top estimate and plan your caching policy accordingly, but be prepared for the top value.

For download visitor bandwidth: number of page views X average page size. Include estimate of images, cs, js, swf etc. Also add the total number of ajax traffic.

For bot traffic: add 100%. Bots are a major part of the "invisible" traffic coming to your web site. This includes google, msn, baidu and yahoo, as well as dozens of others. Also your health checks, your competition, content scrapers and others.

For upload traffic: appx 0.25K per hit. A hit is every request coming your way. EC2 don't charge for it yest, but good to know just in case.

Hardware/servers: Increasing traffic will force you place more servers whether you like it or not. A CDN can help a lot, but cannot alleviate this issue completely, especially if part of the content is dynamic (like from a CMS). Unfortunately there is no formula for this, I can only wish you so much traffic that you need to use a load balancer with a web server cluster.
0
 
LVL 32

Author Closing Comment

by:Daniel Wilson
ID: 39962689
Thanks!
0

Featured Post

Database Solutions Engineer FAQs

In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller single-server environments.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Developer portfolios can be a bit of an enigma—how do you present yourself to employers without burying them in lines of code?  A modern portfolio is more than just work samples, it’s also a statement of how you work.
CTAs encourage people to do something specific to show interest in your company, product or service. Keep reading to learn why CTAs should always be thought of as extremely important, albeit small, sections of websites.
The viewer will learn how to count occurrences of each item in an array.
The goal of the tutorial is to teach the user how to set up your camera and your microphone in Skype.

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question