Solved

How big can a MySQL database grow without performance problems

Posted on 2014-02-06
9
645 Views
Last Modified: 2014-02-08
Hello,
Is there a way to expand a MySQL database beyond OS folder size using InnoDB, MyISam
or any other STORAGE ENGINE?

Oracle tablespaces can grow to just about anything but I get the impression MySQL is limited to the @@datadir location limitations
0
Comment
Question by:Robert Silver
9 Comments
 
LVL 2

Author Comment

by:Robert Silver
Comment Utility
looking at the reference manual I found:
The effective maximum table size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-todate information, be sure to check the documentation specific to your operating system.
  Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use
with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS)
patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and
include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big
files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
For a detailed overview about LFS in Linux, have a look at Andreas Jaeger's Large File Support in Linux
page at http://www.suse.de/~aj/linux_lfs.html.

table of file size limits for mysql
0
 
LVL 2

Author Comment

by:Robert Silver
Comment Utility
I am interested in hearing back if there is another way to organize the data, index and format files e.g
provide a database options file with a list of locations or split up the frm, myi and myd files
perhaps as a set of datafiles much like Oracle does for Oracle table spaces???
0
 
LVL 82

Expert Comment

by:Dave Baldwin
Comment Utility
I'm wondering how large a database you are anticipating.  The file size limits apply to the individual files in the directories for each database.  In theory, you could have at least 2GB frm, myi, and myd files for each table in your database.  If you had 100 tables, that would be 600GB on the most restrictive system you have listed above.  Or 600TB on Win32/NTFS and 1200TB on Linux 2.4+.  That's an awful lot of data.
0
 
LVL 28

Expert Comment

by:FishMonger
Comment Utility
The OS file size limits have very little if anything to do with DB performance.  You can have very poor performance on small databases if they and your queries are not setup properly.

How big of a database are you working with in terms of total disk space?
How many rows are there in the larger tables?
What type of storage engine are you using?
To what level is the DB normalized?
Do you have a primary key in each table?
Do you have indexes and are those indexes setup correctly/efficiently?
Are your queries setup to take advantage of the indexes?
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 2

Author Comment

by:Robert Silver
Comment Utility
Well just a though here what if I have 10 1 TB drives and 5TB of data and image blobs
and I wish to create a MySQL database with say a single table. Not typical but it illustrates my need.  In Oracle  8.x - 11g  I could just add to my tablespace datafies and no worries, but with MySQL
I seem to have only one datafile per table. I guess I am asking is there another Storage Engine besides Innodb that supports distributed data files across my 10 disks ???

I am un aware of such a MySQL solution.

Funny no one had asked this. I know I could just trash the 10 1TB disks and get  one 10Tb disk but my question still stands
0
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
Comment Utility
Depending on your OS and drive array setup, you should be able to bind your individual disks into a logical volume.

Binding them at the hardware level is likely the 'best' in terms of performance but if you cannot, then there should be a software binding option in the OS.
0
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
Comment Utility
in mySQL, using innodb engine, you can create tablespaces as big as you wish:
http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
and add datafiles as you wish.
0
 
LVL 28

Accepted Solution

by:
FishMonger earned 500 total points
Comment Utility
With tables of that size, you should look into partitioning which splits up the table into multiple files.

dev.mysql.com - Chapter 18. Partitioning
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

APEX (Application Express) is used to develop a web application from Oracle. SQL Workshop is one of the tools that comes with Oracle APEX to query or modify the database objects or to make any changes to the structure.
CCModeler offers a way to enter basic information like entities, attributes and relationships and export them as yEd or erviz diagram. It also can import existing Access or SQL Server tables with relationships.
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now