Solved

size of db - performance

Posted on 2013-11-05
5
208 Views
Last Modified: 2013-11-10
I have a db that has grown to 10 GB.  I'm wondering in the long term, how big can a database get before it becomes a concern?  Also is there a performance issue that comes into play as the db grows in size?
0
Comment
Question by:al4629740
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
5 Comments
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625179
A properly partitioned and indexed database can be tens of terabytes and not run into performance issues. But of course, the more data you toss into a database, the more you're going to discover just how good your indexes and queries are written. In other words, it definitely CAN be a performance issue, but the problems can be overcome by tweaking the queries and/or indexes. Physical storage could also become a concern and you may need better harware to support really large databases efficiently.
0
 

Author Comment

by:al4629740
ID: 39625186
What do you mean by "properly partitioned"?
0
 
LVL 8

Assisted Solution

by:virtuadept
virtuadept earned 375 total points
ID: 39625206
You can partition large tables so that the entire table does not have to be stored in the same place or on the same speed/performance of disk. A lot of large databases do this to still have access to historical data but keep from having to reindex millions of rows because you typically only have to re-index the recent stuff.

If you are on Microsoft SQL then you can read about partitioning here:

http://technet.microsoft.com/en-us/library/ms178148(v=SQL.105).aspx
0
 
LVL 10

Assisted Solution

by:PadawanDBA
PadawanDBA earned 125 total points
ID: 39625223
In addition to what virtuadept mentioned, I would also have the concern about the growth of the underlying data files and segmentation.  If the database is autogrowing frequently and there are multiple files, then it's going to be segmented on storage, which is going to yield degraded performance for all operations that have to go to the disk subsystem.
0
 
LVL 8

Accepted Solution

by:
virtuadept earned 375 total points
ID: 39630296
Just wondering if we sufficiently helped answer the question? Also I did find this great article about managing data growth on MS SQL databases. I recommend giving this a good reading or two, as this in depth article is full of excellent advice on monitoring and managing database growth.

https://www.simple-talk.com/sql/database-administration/managing-data-growth-in-sql-server/

Keep in mind that unless you have some kind of automatic data purge, any database that deals in any type of over-time transactional data (data that happens over time, like invoices for a company, or deliveries for a shipping center, or new arrivals for a warehouse, or basically anything, nearly), you are going to experience fairly rapid database growth. The more transactions that you have and the frequency of them will determine just how fast the growth is. And as a business grows, that growth per time period will tend to increase.

Changes to the way that the application handles buisness can also dramatically change database growth. For example, in one of our applications I supported a few years ago we were storing metering measurment data on a daily basis, and then switched to hourly measurement. Well, as you can imagine, that caused our DBAs to panic a little bit when they noticed that the growth of the database jumped up 24 times what it had been the previous month. :-)

So planning and monitoring is key. You don't want to get in a situation where your performance is unacceptable because you've let the tables get too big for the code using those tables. But as I mentioned, these problems are not unsurmountable if you partition and/or index them properly.

Good luck!
0

Featured Post

Space-Age Communications Transitions to DevOps

ViaSat, a global provider of satellite and wireless communications, securely connects businesses, governments, and organizations to the Internet. Learn how ViaSat’s Network Solutions Engineer, drove the transition from a traditional network support to a DevOps-centric model.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows gives you an overview on SQL Server 2016 row level security. You will also get to know the usages of row-level-security and how it works
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question