what is database scalability

Rohit Bajaj
Rohit Bajaj used Ask the Experts™
on
Hi,
I came across a question which says
Design Flipkart Flash sale architecture. 2 million hits and 20k orders in 2 sec. Find out database scalability.  
Now here if we forget about what is flash sale. And some simple application with 2 million hits and 20k orders in 2 sec.
What is this database scalability all about ?
I have made small applications with not so many hits and have always used mysql database.

Thanks
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Software Developer / Linux System Administrator / Managing Director
Commented:
I would imagine that the question relates to the ability of the database to go from "normal" processing on a daily basis and scale (grow to accommodate) the processing requirement that exists during a flash sale. Clearly, someone designing a system will not want to dedicate the resources that a day-sale requires 100% of the time. They would want the architecture to accommodate the increased processing requirement and then drop down to the "normal" daily requirement. It's quite a complex topic in of itself and involves design of appropriate storage systems, processing capability and memory. Often, clustered systems that can activate and deactivate processing nodes as and when needed based on demand would be involved.

Hopefully another expert can provide you with more detail than I am able to - I don't have much involvement with such systems.
nociSoftware Engineer
Distinguished Expert 2018
Commented:
Scalability means that you can design an archtecture that has the ability to adjust to load.
Like if the load doubles and you add double the servers you can handle the load.... if it doubles again and you double the servers again it can still handle the load.

So a design with a single server won't fit in this model,
Not to say that this adjustment is ad-hoc..., just like in January you seen load X, in March you see 1.5*load X..., so in May you can expect 2*load X....
Now you can order extra equipment and add it to the existing infrastructure during April (preferably without downtime) so ya can handle the load in May...

The Built in ability to adjust the scale of the operation is scalability.
(migrating a database to another platform might take a few days...(downtime), so that clearly is not scalable).  So that would rule out a single box design.
For databases you need performance so running in VM's nearly halves the IO capacity compared to bare-metal (some exceptions do exist... f.e. XEN with
dedicated storage/network controllers  per VM).
Therefore you mostly see Clustered bare-metal solutions in large volume solutions.
David FavorFractional CTO
Distinguished Expert 2018
Commented:
Database scalability revolves around disk writes.

For example, given adequate memory, reads will be optimized to run from buffer memory.

Writes will also go to buffer memory, if they're small enough to fit. If not, then you'll end up with physical disk thrash (slowdown) from many writes to physical media.

To pave the way for easy scaling...

1) Reduce number of writes as much as possible.

2) Keep your row lengths small, by creating tables which are only accessed when the specific data is required.

In other words, have many tables, rather than jamming 100s of columns or many long columns into one table, where the table is read/written many times to update different columns.

3) Run mysqltuner periodically + return your MariaDB/MySQL config to match suggestions provided.

4) Only use MariaDB or maybe MySQL 8. Do performance testing with your schemas to see which works best. You may be surprised how well MariaDB out performs MySQL for most workloads.

5) As noci mentioned. Bare Metal will always run faster than VM based installs. Many times orders of magnitude faster, especially when all VMs on a give machine are under load.
Tomas Helgi JohannssonDatabase Administrator / Software Engineer
Commented:
Hi,

This depends on the overall architecture. Modern databases can handle considerable amount of concurrent connections on a single host before the resources regarding CPU/memory/disk IO/network throughput exhausts.
Depending on your requirements and the chosen architecture you could scale the database either vertically where you add resources to the database host. Or scale the database horizontally where you start another database host (an exact copy of the database) in a cluster fashion with a load balancing mechanism in front (or built in the database if available).
Oracle RAC, Db2 pureScale and MariaDB Galera cluster are examples of a good horizontal database systems where all nodes of the database clusters can handle select,insert, update and delete activity (aka CRUD in a multi-master database environment) while the data is replicated in real time between nodes (hosts) and adding nodes to the cluster is "just few mouse-clicks away" ;)

And as you are have used Mysql before then the transition to MariaDB Galera cluster is very simple and easy task. :)

Regards,
    Tomas Helgi
Most Valuable Expert 2012
Distinguished Expert 2018

Commented:
>>Database scalability revolves around disk writes.

Sorry but I have to disagree.  It has to do with ALL resources involved as the application/usage grows.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial