Solved

How big can a MySQL database grow without performance problems

Posted on 2014-02-06
9
655 Views
Last Modified: 2014-02-08
Hello,
Is there a way to expand a MySQL database beyond OS folder size using InnoDB, MyISam
or any other STORAGE ENGINE?

Oracle tablespaces can grow to just about anything but I get the impression MySQL is limited to the @@datadir location limitations
0
Comment
Question by:Robert Silver
9 Comments
 
LVL 2

Author Comment

by:Robert Silver
ID: 39840086
looking at the reference manual I found:
The effective maximum table size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-todate information, be sure to check the documentation specific to your operating system.
  Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use
with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS)
patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and
include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big
files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
For a detailed overview about LFS in Linux, have a look at Andreas Jaeger's Large File Support in Linux
page at http://www.suse.de/~aj/linux_lfs.html.

table of file size limits for mysql
0
 
LVL 2

Author Comment

by:Robert Silver
ID: 39840102
I am interested in hearing back if there is another way to organize the data, index and format files e.g
provide a database options file with a list of locations or split up the frm, myi and myd files
perhaps as a set of datafiles much like Oracle does for Oracle table spaces???
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 39841191
I'm wondering how large a database you are anticipating.  The file size limits apply to the individual files in the directories for each database.  In theory, you could have at least 2GB frm, myi, and myd files for each table in your database.  If you had 100 tables, that would be 600GB on the most restrictive system you have listed above.  Or 600TB on Win32/NTFS and 1200TB on Linux 2.4+.  That's an awful lot of data.
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 28

Expert Comment

by:FishMonger
ID: 39841863
The OS file size limits have very little if anything to do with DB performance.  You can have very poor performance on small databases if they and your queries are not setup properly.

How big of a database are you working with in terms of total disk space?
How many rows are there in the larger tables?
What type of storage engine are you using?
To what level is the DB normalized?
Do you have a primary key in each table?
Do you have indexes and are those indexes setup correctly/efficiently?
Are your queries setup to take advantage of the indexes?
0
 
LVL 2

Author Comment

by:Robert Silver
ID: 39841912
Well just a though here what if I have 10 1 TB drives and 5TB of data and image blobs
and I wish to create a MySQL database with say a single table. Not typical but it illustrates my need.  In Oracle  8.x - 11g  I could just add to my tablespace datafies and no worries, but with MySQL
I seem to have only one datafile per table. I guess I am asking is there another Storage Engine besides Innodb that supports distributed data files across my 10 disks ???

I am un aware of such a MySQL solution.

Funny no one had asked this. I know I could just trash the 10 1TB disks and get  one 10Tb disk but my question still stands
0
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
ID: 39841947
Depending on your OS and drive array setup, you should be able to bind your individual disks into a logical volume.

Binding them at the hardware level is likely the 'best' in terms of performance but if you cannot, then there should be a software binding option in the OS.
0
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39842004
in mySQL, using innodb engine, you can create tablespaces as big as you wish:
http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
and add datafiles as you wish.
0
 
LVL 28

Accepted Solution

by:
FishMonger earned 500 total points
ID: 39842096
With tables of that size, you should look into partitioning which splits up the table into multiple files.

dev.mysql.com - Chapter 18. Partitioning
0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Creating and Managing Databases with phpMyAdmin in cPanel.
Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

815 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now