Avatar of 25112
25112
 asked on

partitioning database after decade growth

pl see chart.{it includes data+index together} (SQL 2008R2)

after 10 years, db nearing 600GB.. what are best standards to break this for performance (perhaps by years of data, since all records have timestamp)

and reports are often done on 8 years of data.
600.png
Microsoft SQL Server 2008DatabasesMongoDB

Avatar of undefined
Last Comment
25112

8/22/2022 - Mon
Vitor Montalvão

You gave only the database size. Partitioning is only for tables so you should start to filter only for large tables. For example, how many tables do you have that are more than 20GB size?
Instead of partitioning you can also move the old data to an archived database. This will be good if you know that older data won't be needed anymore but you still have it archived just in case of future need.
ste5an

Hard to tell as already Vitor said..

But partitioning alone is normally not the solution. Also distribute your data over multiple file groups. This allows you to deactivate old data without much hassle (it's then a simple meta operation instead of a delete). The only thing you need to consider is a rolling window.

8 years of data
A fixed time frame or rolling window? In the later case you need to consider one partition per year and a rolling window approach to add new years.
25112

ASKER
1.
>>how many tables do you have that are more than 20GB size?
five)
171.449218 (in GB)
85.838867
48.428710
34.099609
32.537109

2)
archived database.

basically make new databases right?
database_2003
database_2004 etc? (and delete 2003/2004 data)

3)
distribute your data over multiple file groups. (meta)
The database will be still big/same, right? backups/restore should take same time? will reports be smart to only hit the needed filegroups and hence less IO?

4)
rolling window
yes.
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
SOLUTION
ste5an

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Vitor Montalvão

1.five)
All those 5 tables has at least a date or datetime field?

basically make new databases right?
 database_2003
 database_2004 etc? (and delete 2003/2004 data)
I wanted to suggest only a single database for archiving. You just need to be sure about which old data to be move into archive (data that you're almost sure it won't be accessed anymore).
25112

ASKER
>>All those 5 tables has at least a date or datetime field?
yes; all tables are date-based

>>I wanted to suggest only a single database for archiving.
thanks. Is this in addition to possible to partitioning (partition function as recommended in 42059344)or an alternative solution?

thanks for partial backups idea.

with the 5 tablesize i shared, would you recommend start only with the one 171GB or all 5?
ASKER CERTIFIED SOLUTION
Vitor Montalvão

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
25112

ASKER
thank you,
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.