How do I prevent an ETL, that populates cubes by deleting all data in a Star Schema database from scratch each and every time, and instead update just changes?

We have an ETL process that populates a star schema database and then proceeds to create cubes. Problem being its blowing out the star schema ad re doing everything for scratch every single time. Which is incredibly time consuming. We are using SQL Server SSAS 2014 Standard. Is there a way not to do this anymore and just update the information that has changed?
Paul MaurielloSoftware Programmer Developer Analyst EngineerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jim HornMicrosoft SQL Server Developer, Architect, and AuthorCommented:
Define 'ETL Process'.  In SSIS an incrimental load is handled via a MERGE component that can detect rows that are inserts/updates/deletes(?) and then take appropriate action in a database.  Warning:  Merge's are very difficult to set up.

The easier way would be to (if the files are from an outside party) negotiate with the technical contacts responsible for these files so that they only send you rows that change, or (if the source is an internal database) make sure that both source and target have a 'Last Updated By' datetime column such that only the rows where the 'Last Updated By' is greater than the previous run go through your ETL, which means they are all changes.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Paul MaurielloSoftware Programmer Developer Analyst EngineerAuthor Commented:
Thank you :)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
SSAS

From novice to tech pro — start learning today.