Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 691
  • Last Modified:

How big can a MySQL database grow without performance problems

Hello,
Is there a way to expand a MySQL database beyond OS folder size using InnoDB, MyISam
or any other STORAGE ENGINE?

Oracle tablespaces can grow to just about anything but I get the impression MySQL is limited to the @@datadir location limitations
0
Robert Silver
Asked:
Robert Silver
1 Solution
 
Robert SilverSr. Software EngineerAuthor Commented:
looking at the reference manual I found:
The effective maximum table size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-todate information, be sure to check the documentation specific to your operating system.
  Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use
with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS)
patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and
include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big
files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
For a detailed overview about LFS in Linux, have a look at Andreas Jaeger's Large File Support in Linux
page at http://www.suse.de/~aj/linux_lfs.html.

table of file size limits for mysql
0
 
Robert SilverSr. Software EngineerAuthor Commented:
I am interested in hearing back if there is another way to organize the data, index and format files e.g
provide a database options file with a list of locations or split up the frm, myi and myd files
perhaps as a set of datafiles much like Oracle does for Oracle table spaces???
0
 
Dave BaldwinFixer of ProblemsCommented:
I'm wondering how large a database you are anticipating.  The file size limits apply to the individual files in the directories for each database.  In theory, you could have at least 2GB frm, myi, and myd files for each table in your database.  If you had 100 tables, that would be 600GB on the most restrictive system you have listed above.  Or 600TB on Win32/NTFS and 1200TB on Linux 2.4+.  That's an awful lot of data.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
FishMongerCommented:
The OS file size limits have very little if anything to do with DB performance.  You can have very poor performance on small databases if they and your queries are not setup properly.

How big of a database are you working with in terms of total disk space?
How many rows are there in the larger tables?
What type of storage engine are you using?
To what level is the DB normalized?
Do you have a primary key in each table?
Do you have indexes and are those indexes setup correctly/efficiently?
Are your queries setup to take advantage of the indexes?
0
 
Robert SilverSr. Software EngineerAuthor Commented:
Well just a though here what if I have 10 1 TB drives and 5TB of data and image blobs
and I wish to create a MySQL database with say a single table. Not typical but it illustrates my need.  In Oracle  8.x - 11g  I could just add to my tablespace datafies and no worries, but with MySQL
I seem to have only one datafile per table. I guess I am asking is there another Storage Engine besides Innodb that supports distributed data files across my 10 disks ???

I am un aware of such a MySQL solution.

Funny no one had asked this. I know I could just trash the 10 1TB disks and get  one 10Tb disk but my question still stands
0
 
slightwv (䄆 Netminder) Commented:
Depending on your OS and drive array setup, you should be able to bind your individual disks into a logical volume.

Binding them at the hardware level is likely the 'best' in terms of performance but if you cannot, then there should be a software binding option in the OS.
0
 
Guy Hengel [angelIII / a3]Billing EngineerCommented:
in mySQL, using innodb engine, you can create tablespaces as big as you wish:
http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
and add datafiles as you wish.
0
 
FishMongerCommented:
With tables of that size, you should look into partitioning which splits up the table into multiple files.

dev.mysql.com - Chapter 18. Partitioning
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now