Solved

How big can a MySQL database grow without performance problems

Posted on 2014-02-06
9
649 Views
Last Modified: 2014-02-08
Hello,
Is there a way to expand a MySQL database beyond OS folder size using InnoDB, MyISam
or any other STORAGE ENGINE?

Oracle tablespaces can grow to just about anything but I get the impression MySQL is limited to the @@datadir location limitations
0
Comment
Question by:Robert Silver
9 Comments
 
LVL 2

Author Comment

by:Robert Silver
ID: 39840086
looking at the reference manual I found:
The effective maximum table size for MySQL databases is usually determined by operating system
constraints on file sizes, not by MySQL internal limits. The following table lists some examples of operating
system file-size limits. This is only a rough guide and is not intended to be definitive. For the most up-todate information, be sure to check the documentation specific to your operating system.
  Windows users, please note that FAT and VFAT (FAT32) are not considered suitable for production use
with MySQL. Use NTFS instead.
On Linux 2.2, you can get MyISAM tables larger than 2GB in size by using the Large File Support (LFS)
patch for the ext2 file system. Most current Linux distributions are based on kernel 2.4 or higher and
include all the required LFS patches. On Linux 2.4, patches also exist for ReiserFS to get support for big
files (up to 2TB). With JFS and XFS, petabyte and larger files are possible on Linux.
For a detailed overview about LFS in Linux, have a look at Andreas Jaeger's Large File Support in Linux
page at http://www.suse.de/~aj/linux_lfs.html.

table of file size limits for mysql
0
 
LVL 2

Author Comment

by:Robert Silver
ID: 39840102
I am interested in hearing back if there is another way to organize the data, index and format files e.g
provide a database options file with a list of locations or split up the frm, myi and myd files
perhaps as a set of datafiles much like Oracle does for Oracle table spaces???
0
 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 39841191
I'm wondering how large a database you are anticipating.  The file size limits apply to the individual files in the directories for each database.  In theory, you could have at least 2GB frm, myi, and myd files for each table in your database.  If you had 100 tables, that would be 600GB on the most restrictive system you have listed above.  Or 600TB on Win32/NTFS and 1200TB on Linux 2.4+.  That's an awful lot of data.
0
 
LVL 28

Expert Comment

by:FishMonger
ID: 39841863
The OS file size limits have very little if anything to do with DB performance.  You can have very poor performance on small databases if they and your queries are not setup properly.

How big of a database are you working with in terms of total disk space?
How many rows are there in the larger tables?
What type of storage engine are you using?
To what level is the DB normalized?
Do you have a primary key in each table?
Do you have indexes and are those indexes setup correctly/efficiently?
Are your queries setup to take advantage of the indexes?
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 
LVL 2

Author Comment

by:Robert Silver
ID: 39841912
Well just a though here what if I have 10 1 TB drives and 5TB of data and image blobs
and I wish to create a MySQL database with say a single table. Not typical but it illustrates my need.  In Oracle  8.x - 11g  I could just add to my tablespace datafies and no worries, but with MySQL
I seem to have only one datafile per table. I guess I am asking is there another Storage Engine besides Innodb that supports distributed data files across my 10 disks ???

I am un aware of such a MySQL solution.

Funny no one had asked this. I know I could just trash the 10 1TB disks and get  one 10Tb disk but my question still stands
0
 
LVL 76

Expert Comment

by:slightwv (䄆 Netminder)
ID: 39841947
Depending on your OS and drive array setup, you should be able to bind your individual disks into a logical volume.

Binding them at the hardware level is likely the 'best' in terms of performance but if you cannot, then there should be a software binding option in the OS.
0
 
LVL 142

Expert Comment

by:Guy Hengel [angelIII / a3]
ID: 39842004
in mySQL, using innodb engine, you can create tablespaces as big as you wish:
http://dev.mysql.com/doc/refman/5.1/en/create-tablespace.html
and add datafiles as you wish.
0
 
LVL 28

Accepted Solution

by:
FishMonger earned 500 total points
ID: 39842096
With tables of that size, you should look into partitioning which splits up the table into multiple files.

dev.mysql.com - Chapter 18. Partitioning
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

APEX (Application Express) is used to develop a web application from Oracle. SQL Workshop is one of the tools that comes with Oracle APEX to query or modify the database objects or to make any changes to the structure.
These days, all we hear about hacktivists took down so and so websites and retrieved thousands of user’s data. One of the techniques to get unauthorized access to database is by performing SQL injection. This article is quite lengthy which gives bas…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now