Infrastructure Design suggestions for a Multi-Server Aplication


My company is working on developing a could-based business management solution.

Our developers have asked for suggestions on the best way to implement a multi-server infrastructure.

Instead of simply running a website, or server application on one box, it would need to run on many different servers to handle the processing power / db queries performed by many users.

For example, Everyone goes to, yet there isnt one central box that handles it the entire site.

Could someone roughly explain the theory behind how such sites like Facebook operate?

My ROUGH idea is something like this...

1. Go to
2. Script determines which geographical location you're in
3. You're routed to the closest NLB in the FB network.
4. you're then routed to the server inside the network which has the least number of users on it.

Please note that i only use Facebook as an example.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

The point is that you create one entry box in a cluster. But you will cut out the calculations process in a cloud like eucalyptus for linux. Its like in the old days when you rented cpu time in a data center. So the design of the databse is the thing you have to do first. By a small system you can log which part of your database is accessed by clients. and this part you generate new as a single database and replement the datas back in a link for your applications.
VCSLIAuthor Commented:
I think what you're suggesting is that we have a different box for each part of the database.

This isn't ideal for this project as we need everything stored in one place. Users must be able to access all data their entitled to regardless of which box they're on.

Go to, chances are you're not routed to server 32 on Rack B twice... Each time it's a different box handeling your session however you can still access all the information you're meant to.

The point is that the users are all on the same box but your databases not. So the routing problem is not a problem at all see here:
Acronis Data Cloud 7.8 Enhances Cyber Protection

A closer look at five essential enhancements that benefit end-users and help MSPs take their cloud data protection business further.

VCSLIAuthor Commented:
Are you sating that a massive website like FB has "ONE" server which all users use?

Like one physical box?

How is that possible?
You can break your application into processing and data.  It's usually fairly easy to divide up the processing into a collection of different servers - e.g. each server assembling the web page for or  As you say, you could select the server using some sort of load balancing algorithm to pick a lightly loaded server and that could even be done geographically if your site is large enough and global.  That process isn't too complex.

The harder problem is what to do about the data.

In some applications, the processing servers only need access to a small portion of the entire database.  But that's not the case in your application and it's not the case in facebook and most others.  

In those situations one approach is to take a normal relational database (e.g. MySQL or Oracle) and shard it into a series of smaller databases.  E.g. If your users log into with a username, you could put all users whose username began with "a" on database 1, "b" on database 2 and so on.  This would give you 26 database servers each processing some share of the load.  (In practice you want to be smarter about distributing the load - but this should give you the idea).  When you need the data for user "andy" you know to look for it on server 1.  Ebay for instance is built around this sort of model, since they require database like behavior from an auction (all bids happen as a transaction etc.).  

The other option is adopt a "NoSQL" framework.  These frameworks are specifically designed to scale to huge levels, but they do so by giving you much less sophisticated access to your data than a full relational database allows.  Things like database joins are usually not supported.  They basically are more like file systems than databases, but with great support for scaling to many boxes and the ability to support indexing (like a database) so you can find the files.  There are lots of these frameworks around - each with strengths and weaknesses.  Check out Hadoop (,  Cassandra (which facebook actually developed) and Google's bigtable ( to name just a few.

As you'll quickly realize as you get into this is that (a) it's a big problem and challenging to solve at really large scale and (b) nobody has really "solved" it yet - there's lots of different options and each comes with pluses and minuses.  Do a lot of research before you jump in if you really need your application to get really big.  But at the same time don't go crazy.  If you can estimate your number of users and compute the expected capacity of a single server, you may find you only need 10 servers for your first 5 million users say and the solutions for 10 servers can be a lot simpler than the solution for 1,000 servers.

I hope this help get you started,


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
VCSLIAuthor Commented:

I sincerely appreciate your advice on this. Thank you for taking the time to write such an extensive response. I will forward this to my developers and see what they say.

In the mean time i'd like to leave this thread open and see if i can get any more ideas.

Thank you very much once again.
You're very welcome.  All the best in learning about this fascinating world.

VCSLIAuthor Commented:
Thank you for the help!!!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.