Managed service contracts are great - when they're making you money. Yes, you’re getting paid monthly, but is it actually profitable? Learn to calculate your hourly overhead burden so you can master your IT services pricing strategy.
partitioning database after decade growth
pl see chart.{it includes data+index together} (SQL 2008R2)
after 10 years, db nearing 600GB.. what are best standards to break this for performance (perhaps by years of data, since all records have timestamp)
and reports are often done on 8 years of data.
600.png
after 10 years, db nearing 600GB.. what are best standards to break this for performance (perhaps by years of data, since all records have timestamp)
and reports are often done on 8 years of data.
600.png
Asked:
Who is Participating?
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get this solution by purchasing an Individual license!
Start your 7-day free trial.
I wear a lot of hats...
"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.
Hard to tell as already Vitor said..
But partitioning alone is normally not the solution. Also distribute your data over multiple file groups. This allows you to deactivate old data without much hassle (it's then a simple meta operation instead of a delete). The only thing you need to consider is a rolling window.
But partitioning alone is normally not the solution. Also distribute your data over multiple file groups. This allows you to deactivate old data without much hassle (it's then a simple meta operation instead of a delete). The only thing you need to consider is a rolling window.
8 years of dataA fixed time frame or rolling window? In the later case you need to consider one partition per year and a rolling window approach to add new years.
1.
>>how many tables do you have that are more than 20GB size?
five)
171.449218 (in GB)
85.838867
48.428710
34.099609
32.537109
2)
archived database.
basically make new databases right?
database_2003
database_2004 etc? (and delete 2003/2004 data)
3)
distribute your data over multiple file groups. (meta)
The database will be still big/same, right? backups/restore should take same time? will reports be smart to only hit the needed filegroups and hence less IO?
4)
rolling window
yes.
>>how many tables do you have that are more than 20GB size?
five)
171.449218 (in GB)
85.838867
48.428710
34.099609
32.537109
2)
archived database.
basically make new databases right?
database_2003
database_2004 etc? (and delete 2003/2004 data)
3)
distribute your data over multiple file groups. (meta)
The database will be still big/same, right? backups/restore should take same time? will reports be smart to only hit the needed filegroups and hence less IO?
4)
rolling window
yes.
3) Yes and no. A full backup will need the same time. But if the data in "older" partitions don't change, then you can do partial backups and restores over file groups.
See also: How to: Back Up Files and Filegroups (Transact-SQL)
The problem with separate databases is that you need to consolidate the data manually (harcoded) by adding views like
Querying a partitioned table uses the partition function to select only the needed partition and is thus faster.
See also: How to: Back Up Files and Filegroups (Transact-SQL)
The problem with separate databases is that you need to consolidate the data manually (harcoded) by adding views like
SELECT *
FROM database_2003.dbo.tableA
UNION ALL
SELECT *
FROM database_2004.dbo.tableA
UNION ALL -- ..
SELECT *
FROM database_2017.dbo.tableA
Querying a partitioned table uses the partition function to select only the needed partition and is thus faster.
1.five)All those 5 tables has at least a date or datetime field?
basically make new databases right?I wanted to suggest only a single database for archiving. You just need to be sure about which old data to be move into archive (data that you're almost sure it won't be accessed anymore).
database_2003
database_2004 etc? (and delete 2003/2004 data)
>>All those 5 tables has at least a date or datetime field?
yes; all tables are date-based
>>I wanted to suggest only a single database for archiving.
thanks. Is this in addition to possible to partitioning (partition function as recommended in 42059344)or an alternative solution?
thanks for partial backups idea.
with the 5 tablesize i shared, would you recommend start only with the one 171GB or all 5?
yes; all tables are date-based
>>I wanted to suggest only a single database for archiving.
thanks. Is this in addition to possible to partitioning (partition function as recommended in 42059344)or an alternative solution?
thanks for partial backups idea.
with the 5 tablesize i shared, would you recommend start only with the one 171GB or all 5?
Partitioning or archiving is up to you but I would say that if the old data is keep being accessed regularly then Partitioning should be a better choice. If the old data is really not needed then you can go for an Archive solution.
Experts Exchange Solution brought to you by
Your issues matter to us.
Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.
Start your 7-day free trialIt's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server 2008
From novice to tech pro — start learning today.
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get this solution by purchasing an Individual license!
Start your 7-day free trial.
Instead of partitioning you can also move the old data to an archived database. This will be good if you know that older data won't be needed anymore but you still have it archived just in case of future need.