Solved

size of db - performance

Posted on 2013-11-05
5
202 Views
Last Modified: 2013-11-10
I have a db that has grown to 10 GB.  I'm wondering in the long term, how big can a database get before it becomes a concern?  Also is there a performance issue that comes into play as the db grows in size?
0
Comment
Question by:al4629740
  • 3
5 Comments
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625179
A properly partitioned and indexed database can be tens of terabytes and not run into performance issues. But of course, the more data you toss into a database, the more you're going to discover just how good your indexes and queries are written. In other words, it definitely CAN be a performance issue, but the problems can be overcome by tweaking the queries and/or indexes. Physical storage could also become a concern and you may need better harware to support really large databases efficiently.
0
 

Author Comment

by:al4629740
ID: 39625186
What do you mean by "properly partitioned"?
0
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625206
You can partition large tables so that the entire table does not have to be stored in the same place or on the same speed/performance of disk. A lot of large databases do this to still have access to historical data but keep from having to reindex millions of rows because you typically only have to re-index the recent stuff.

If you are on Microsoft SQL then you can read about partitioning here:

http://technet.microsoft.com/en-us/library/ms178148(v=SQL.105).aspx
0
 
LVL 10

Assisted Solution

by:PadawanDBA
PadawanDBA earned 125 total points
ID: 39625223
In addition to what virtuadept mentioned, I would also have the concern about the growth of the underlying data files and segmentation.  If the database is autogrowing frequently and there are multiple files, then it's going to be segmented on storage, which is going to yield degraded performance for all operations that have to go to the disk subsystem.
0
 
LVL 8

Accepted Solution

by:
virtuadept earned 375 total points
ID: 39630296
Just wondering if we sufficiently helped answer the question? Also I did find this great article about managing data growth on MS SQL databases. I recommend giving this a good reading or two, as this in depth article is full of excellent advice on monitoring and managing database growth.

https://www.simple-talk.com/sql/database-administration/managing-data-growth-in-sql-server/

Keep in mind that unless you have some kind of automatic data purge, any database that deals in any type of over-time transactional data (data that happens over time, like invoices for a company, or deliveries for a shipping center, or new arrivals for a warehouse, or basically anything, nearly), you are going to experience fairly rapid database growth. The more transactions that you have and the frequency of them will determine just how fast the growth is. And as a business grows, that growth per time period will tend to increase.

Changes to the way that the application handles buisness can also dramatically change database growth. For example, in one of our applications I supported a few years ago we were storing metering measurment data on a daily basis, and then switched to hourly measurement. Well, as you can imagine, that caused our DBAs to panic a little bit when they noticed that the growth of the database jumped up 24 times what it had been the previous month. :-)

So planning and monitoring is key. You don't want to get in a situation where your performance is unacceptable because you've let the tables get too big for the code using those tables. But as I mentioned, these problems are not unsurmountable if you partition and/or index them properly.

Good luck!
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Using examples as well as descriptions, and references to Books Online, show the documentation available for date manipulation functions and by using a select few of these functions, show how date based data can be manipulated with these functions.

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question