Solved

How do I estimate the bandwidth requirements of a conceptual website?

Posted on 2014-03-18
8
244 Views
Last Modified: 2014-03-28
I'm supposed to come up with cost estimates ... at 100k users ... at 1 million users.  I've estimated the hardware ... also the Amazon EC2 instances if we go that route.

But how do I estimate the bandwidth?  I've tried measuring bandwidth I use in a given time using a "comparable" website (using Chrome's developer tools).  My extrapolations have an "average" user using 480 MB/month from the regular server and 1.4 GB/month from the CDN.  Cost calculations at that level are crazy high.

What is a good estimation method?  Or do some of the big players publish their bandwidth usage?
0
Comment
Question by:Daniel Wilson
8 Comments
 
LVL 58

Expert Comment

by:Gary
ID: 39937245
Your figures seem crazy for one user but...
What are they downloading? Are you accounting for cached content?
0
 
LVL 25

Expert Comment

by:Fred Marshall
ID: 39937340
Have you considered cached content in the context of an edge provider like Akamai?
They might even be able to help you answer your question - or at least tell you that it's too small for them or .....
0
 
LVL 32

Author Comment

by:Daniel Wilson
ID: 39937582
My "similar" site is Facebook.  I'm accumulating the MB's that Chrome's Developer Tools say are downloaded ... which appears to be measuring what's actually downloaded, not what's cached on the browser.

But I agree my figures seem crazy for a single user.  I really suspect my methodology is SERIOUSLY flawed.

How SHOULD I be estimating the usage?
0
 
LVL 58

Expert Comment

by:Gary
ID: 39937609
Initial load count everything, subsequent loads count only the html page size.
You will have to guestimate the number of pages they will visit and what if any new content is on those pages e.g. images.

For example if the initial load is 500kb including all css, js, common images etc then subsequent pages are likely going to be around 10kb (or less) as all that is being downloaded is the HTML.  If you said some of those pages had a few extra images then just add a rough figure to cover those images to the base html size.
And remember once the HTML is downloaded it won't be downloaded again unless it is changed.
If you do have constantly changing pages then think about implementing ajax to update that content.
(Based on a 30+ day cache)
Even comparing to Facebook where you might have real time communication you are only talking about bits and bytes.
If they visited every day for a month and visited the same 5 pages you are likely only talking a few Mb's a month.
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 32

Author Comment

by:Daniel Wilson
ID: 39937867
Gary, I have the strong feeling you're right.

Can you suggest how to verify or quantify that?
0
 
LVL 58

Accepted Solution

by:
Gary earned 300 total points
ID: 39937877
0
 
LVL 33

Assisted Solution

by:shalomc
shalomc earned 200 total points
ID: 39942771
A small methodological correction:
It is wrong to assume that once an image is loaded is will always be served from browser cache unless changed.
Subsequent loads of a page may or may not load all or part of the page, images, css etc.
It depends on the caching policy of the object, on the expiry date, on the etag value, and on the client.
Create a top estimate and plan your caching policy accordingly, but be prepared for the top value.

For download visitor bandwidth: number of page views X average page size. Include estimate of images, cs, js, swf etc. Also add the total number of ajax traffic.

For bot traffic: add 100%. Bots are a major part of the "invisible" traffic coming to your web site. This includes google, msn, baidu and yahoo, as well as dozens of others. Also your health checks, your competition, content scrapers and others.

For upload traffic: appx 0.25K per hit. A hit is every request coming your way. EC2 don't charge for it yest, but good to know just in case.

Hardware/servers: Increasing traffic will force you place more servers whether you like it or not. A CDN can help a lot, but cannot alleviate this issue completely, especially if part of the content is dynamic (like from a CMS). Unfortunately there is no formula for this, I can only wish you so much traffic that you need to use a load balancer with a web server cluster.
0
 
LVL 32

Author Closing Comment

by:Daniel Wilson
ID: 39962689
Thanks!
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why do we like using grid based layouts in website design? Let's look at the live examples of websites and compare them to grid based WordPress themes.
Developer portfolios can be a bit of an enigma—how do you present yourself to employers without burying them in lines of code?  A modern portfolio is more than just work samples, it’s also a statement of how you work.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

30 Experts available now in Live!

Get 1:1 Help Now