eCommerce Website System Design

This question is beyond the scope of a Q&A forum.
Furthermore this sounds very much like an exam question.

The design doesn't really need to be very complicated - you'll want a pair of firewalls, at least 2 web servers and at least 2 database servers.  (The 2nd servers in each layer provide some level of redundancy - for more redundancy/performance you just increase the number of servers in each layer).

Since the design is Java + MySQL the OS will be a flavor of Linux.  You wouldn't want to run on Windows Server with those language choices.

Given the nature of the question there's really no way to answer things like "CPU and Memory choices".  You could guess, but there's almost nothing in the question to indicate the rate of requests per second - which is what you'd really need to know to estimate the servers.

Saying the database will grow to 100GB in a year means almost nothing - that could be done with virtually no load on the database or with massive database load, depending on the application (how many items are being sold and how rapidly is the information about them changing etc.)

These days if you want to have "rapid scalability" you're probably thinking something cloud based - like Amazon's EC2.

Good luck,

Doug

Hi Doug
Sorry for the late reply been very busy.
The scenario is for a customer I am dealing with and was looking to bounce ideas of someone else. It doesn't help that there are no real figures for utilisation etc...
I was thinking a pair of Cisco ASA5510's in high availability mode. Possibility of some Load balancers maybe.
What would be your choice of operating system being Linux? Centos Redhat and why?
Would you go for a 64bit version?
Also whats your reason for not using a Windows you mention you wouldn't want to run it on a Windows Server with those language choices. Whats your reason why?? interested to know.
What OS would you use for hosting the MySQL database would you have it as a cluster or would you setup it up to replicate in some other way.
For rapid scalability I was thinking possibly VMware but also was interested in Amazons cloud based solutions.
What is the database limit for MySQL 5.6?
Could the images of the products be kept within the database or would you recommend something else.
I am picking your brains because typically when asking my account manager he wants me to invest in some major hardware. I am not always confident I get the best return on investment and wanted to know my options.
Any direction or assistance is greatly appreciated.

Keep in mind that I'm a software engineer, not a sys admin but here's the choices I'd recommend:

What would be your choice of operating system being Linux? Centos Redhat and why?

We use CentOS.  It's a solid, stable choice and easy to find engineers who are familiar with it for support.

Would you go for a 64bit version?

That's a must these days.  4GB of RAM (the 32-bit limit) is nothing in the modern world.

Also whats your reason for not using a Windows you mention you wouldn't want to run it on a Windows Server with those language choices

Simplest reason is cost.  There's no reason to pay for a Windows box when you'd gain nothing from it for Java + MySQL.  Plus those are much more extensively used and tested on Linux that Windows these days.  If you said C# and SQL Server...well then it would be Windows :)

What OS would you use for hosting the MySQL database would you have it as a cluster or would you setup it up to replicate in some other way.

We use CentOS for all boxes including database just becomes uniformity simplifies things.
We don't set it up as a formal MySQL cluster because that's not a MySQL strength - we just do multi-master replication (A replicates to B and B replicates to A for each pair of servers).

For rapid scalability I was thinking possibly VMware but also was interested in Amazons cloud based solutions.

Both are fine - just depends on how much hardware you have access to directly and how quickly you think you'll scale.  We actually *don't* use Amazon because we didn't like the I/O throughput (although they've been working on that) but we also don't need to scale that quickly.

What is the database limit for MySQL 5.6?

That I don't know.

Could the images of the products be kept within the database or would you recommend something else.

They could be, but storing images in a SQL database seems inefficient.  Generally easier to just put the images somewhere else (e.g. in Amazon S3 or just on your web server) and store the path in the database.

Good luck,

Doug

Hi Doug
Thanks ever so much for your input.
A few last things to clarify.
With the web servers how would you load balance them I was thinking of using a pair of Jet nexus boxes. Do you think this is over kill what would your approach be?

Hardware wise what would be your choice of spec in regards (Web Servers) of CPU and Memory and with that choice in your professional experience how many active connections could it support and service. Would higher CPU and Memory give me better return of investment. This is assuming I am using Centos 64bit. Also what would you spec for the MySQL boxes.

Would your design have a setup using a SAN to store all the images, videos and other data generated?

Hence why I was thinking having VM's so if they required more resources its easier to increase and limits my down time.

With the web servers how would you load balance them I was thinking of using a pair of Jet nexus boxes. Do you think this is over kill what would your approach be?

We started out just using round robin DNS for load balancing - since that costs nothing.  It's easy to start there and then actual load balancers later if you find you need them and by then you'll know your traffic patterns and can buy the right boxes.

Hardware wise what would be your choice of spec in regards (Web Servers) of CPU and Memory

Web servers are usually easy because they are simple to scale.  They (should) be storing no data, so they just provide computation, which is the simple part of the system to expand - just add another box into the server pool and you get a linear increase in capacity.  So given that we've been running low end boxes (Xeon 2GHz processor, 16GB RAM) and 2 web servers like that can handle thousands of requests per minute.  We use a CDN to keep down load on the web servers, keep costs low (it's cheaper than our data center bandwidth costs) and boost global performance.  So if you do that you can even millions of requests per minute to your users - but because that doesn't come back to your origin servers.

Also what would you spec for the MySQL boxes.

This is where you will spend your money.  Databases unlike web servers can't be scaled out linearly by just adding more of them - at least not unless you design your software carefully.  If they're VMs that helps in turns of flexibility over time - but generally costs significantly in terms of I/O throughput (i.e. MySQL on a VM may only get half the performance of MySQL on the native hardware) so you can flexibility but it costs more.  Specs here are much harder to just guess at.  We run beefy database servers with 96GB of RAM and 24 high speed hard drives.  But our load won't be your load.

Would your design have a setup using a SAN to store all the images, videos and other data generated?

We looked at SANs but they're crazy expensive for the storage you get.  In the end we started with just a simple NFS file server and decided later on that was a big mistake and moved all image storage to Amazon's S3.  It's vastly cheaper and scales much better than buying your own SAN.

Hope that helps,

Doug

Last question promise please could you just clarify a few points your raised

1. You mentioned using round robin DNS for load balancing please could you explain exactly what you mean by this I think I catch your drift just wanted to be double sure.

2. The MySQL box from your technical design am I right in thinking you have 2x of them. Each replicating the data over to the other. If this is the case is the replication real time do you suffer any latency? If the primary MySQL box was to die how do you approach the task of the web servers seeing the other secondary MYSQL box. I am trying to eradicate the single point of failure issue.

3. Do you have any diagrams you wouldn't mind sharing of your typical setup

Very helpful and long explanations as to why items are setup like they are