troubleshooting Question

Freeze portion of datamart

Avatar of jvoconnell
jvoconnell asked on
Microsoft SQL Server
2 Comments1 Solution89 ViewsLast Modified:

We have a data mart that contains data going back 6 to 7 years. Up until now we usually roll off the earliest year. Users have requested not to roll off data any longer as well as going back earlier in time and adding more data. The older data is static. Currently our data mart process is a drop and replace. We use all surrogate keys.

My question is what are the thoughts for the best method for not having to refresh the fact and dimension tables for that older static data so as to keep the build time manageable.

Sorry in advance for the newbie methods we have thought about implementing. One idea was to build  the DM as normal with all the required data to satisfy user requirements. We thought about  finding the max Identity (PK) in each fact and dimension for records that would remain static and also set a flag on those records. Then each month delete the years of data that would need to get refreshed and perhaps reseed the table based on the highest value of the static data. Then just process the newer set of data and appending to the fact and dimensions.

We are probably way off base here. So please excuse the ignorance on our part.

Any insight is appreciated.
Database Analyst

Our community of experts have been thoroughly vetted for their expertise and industry experience.

Join our community to see this answer!
Unlock 1 Answer and 2 Comments.
Start Free Trial
Learn from the best

Network and collaborate with thousands of CTOs, CISOs, and IT Pros rooting for you and your success.

Andrew Hancock - VMware vExpert
See if this solution works for you by signing up for a 7 day free trial.
Unlock 1 Answer and 2 Comments.
Try for 7 days

”The time we save is the biggest benefit of E-E to our team. What could take multiple guys 2 hours or more each to find is accessed in around 15 minutes on Experts Exchange.

-Mike Kapnisakis, Warner Bros