How big can a MySQL database grow without performance problems

Hello,
Is there a way to expand a MySQL database beyond OS folder size using InnoDB, MyISam
or any other STORAGE ENGINE?

Oracle tablespaces can grow to just about anything but I get the impression MySQL is limited to the @@datadir location limitations
LVL 2
Robert SilverSr. Software EngineerAsked:
Who is Participating?
 
FishMongerCommented:
With tables of that size, you should look into partitioning which splits up the table into multiple files.

dev.mysql.com - Chapter 18. Partitioning
0
 
Robert SilverSr. Software EngineerAuthor Commented:
looking at the reference manual I found:
The effective maximum table size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-todate information, be sure to check the documentation specific to your operating system.
  Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use
with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS)
patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and
include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big
files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
For a detailed overview about LFS in Linux, have a look at Andreas Jaeger's Large File Support in Linux
page at http://www.suse.de/~aj/linux_lfs.html.

table of file size limits for mysql
0
 
Robert SilverSr. Software EngineerAuthor Commented:
I am interested in hearing back if there is another way to organize the data, index and format files e.g
provide a database options file with a list of locations or split up the frm, myi and myd files
perhaps as a set of datafiles much like Oracle does for Oracle table spaces???
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
Dave BaldwinFixer of ProblemsCommented:
I'm wondering how large a database you are anticipating.  The file size limits apply to the individual files in the directories for each database.  In theory, you could have at least 2GB frm, myi, and myd files for each table in your database.  If you had 100 tables, that would be 600GB on the most restrictive system you have listed above.  Or 600TB on Win32/NTFS and 1200TB on Linux 2.4+.  That's an awful lot of data.
0
 
FishMongerCommented:
The OS file size limits have very little if anything to do with DB performance.  You can have very poor performance on small databases if they and your queries are not setup properly.

How big of a database are you working with in terms of total disk space?
How many rows are there in the larger tables?
What type of storage engine are you using?
To what level is the DB normalized?
Do you have a primary key in each table?
Do you have indexes and are those indexes setup correctly/efficiently?
Are your queries setup to take advantage of the indexes?
0
 
Robert SilverSr. Software EngineerAuthor Commented:
Well just a though here what if I have 10 1 TB drives and 5TB of data and image blobs
and I wish to create a MySQL database with say a single table. Not typical but it illustrates my need.  In Oracle  8.x - 11g  I could just add to my tablespace datafies and no worries, but with MySQL
I seem to have only one datafile per table. I guess I am asking is there another Storage Engine besides Innodb that supports distributed data files across my 10 disks ???

I am un aware of such a MySQL solution.

Funny no one had asked this. I know I could just trash the 10 1TB disks and get  one 10Tb disk but my question still stands
0
 
slightwv (䄆 Netminder) Commented:
Depending on your OS and drive array setup, you should be able to bind your individual disks into a logical volume.

Binding them at the hardware level is likely the 'best' in terms of performance but if you cannot, then there should be a software binding option in the OS.
0
 
Guy Hengel [angelIII / a3]Billing EngineerCommented:
in mySQL, using innodb engine, you can create tablespaces as big as you wish:
http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
and add datafiles as you wish.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.