Solved

size of db - performance

Posted on 2013-11-05
5
193 Views
Last Modified: 2013-11-10
I have a db that has grown to 10 GB.  I'm wondering in the long term, how big can a database get before it becomes a concern?  Also is there a performance issue that comes into play as the db grows in size?
0
Comment
Question by:al4629740
  • 3
5 Comments
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625179
A properly partitioned and indexed database can be tens of terabytes and not run into performance issues. But of course, the more data you toss into a database, the more you're going to discover just how good your indexes and queries are written. In other words, it definitely CAN be a performance issue, but the problems can be overcome by tweaking the queries and/or indexes. Physical storage could also become a concern and you may need better harware to support really large databases efficiently.
0
 

Author Comment

by:al4629740
ID: 39625186
What do you mean by "properly partitioned"?
0
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625206
You can partition large tables so that the entire table does not have to be stored in the same place or on the same speed/performance of disk. A lot of large databases do this to still have access to historical data but keep from having to reindex millions of rows because you typically only have to re-index the recent stuff.

If you are on Microsoft SQL then you can read about partitioning here:

http://technet.microsoft.com/en-us/library/ms178148(v=SQL.105).aspx
0
 
LVL 10

Assisted Solution

by:PadawanDBA
PadawanDBA earned 125 total points
ID: 39625223
In addition to what virtuadept mentioned, I would also have the concern about the growth of the underlying data files and segmentation.  If the database is autogrowing frequently and there are multiple files, then it's going to be segmented on storage, which is going to yield degraded performance for all operations that have to go to the disk subsystem.
0
 
LVL 8

Accepted Solution

by:
virtuadept earned 375 total points
ID: 39630296
Just wondering if we sufficiently helped answer the question? Also I did find this great article about managing data growth on MS SQL databases. I recommend giving this a good reading or two, as this in depth article is full of excellent advice on monitoring and managing database growth.

https://www.simple-talk.com/sql/database-administration/managing-data-growth-in-sql-server/

Keep in mind that unless you have some kind of automatic data purge, any database that deals in any type of over-time transactional data (data that happens over time, like invoices for a company, or deliveries for a shipping center, or new arrivals for a warehouse, or basically anything, nearly), you are going to experience fairly rapid database growth. The more transactions that you have and the frequency of them will determine just how fast the growth is. And as a business grows, that growth per time period will tend to increase.

Changes to the way that the application handles buisness can also dramatically change database growth. For example, in one of our applications I supported a few years ago we were storing metering measurment data on a daily basis, and then switched to hourly measurement. Well, as you can imagine, that caused our DBAs to panic a little bit when they noticed that the growth of the database jumped up 24 times what it had been the previous month. :-)

So planning and monitoring is key. You don't want to get in a situation where your performance is unacceptable because you've let the tables get too big for the code using those tables. But as I mentioned, these problems are not unsurmountable if you partition and/or index them properly.

Good luck!
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Having an SQL database can be a big investment for a small company. Hardware, setup and of course, the price of software all add up to a big bill that some companies may not be able to absorb.  Luckily, there is a free version SQL Express, but does …
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.
Viewers will learn how to use the SELECT statement in SQL and will be exposed to the many uses the SELECT statement has.

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question