SSIS package

Hello Expert:

Since SSIS is one of SQL Server 2005's feature, I post my question here. I am sorry if it is not fitting in this topic area.

I have an assignment for creating a SSIS package which loading around 20 million rows data into a fact table in the data mart. My question is: what is the better solution when the package failed duing the loading data period (let say, 5 million rows data has been loaded in the fact table when the SSIS package(ETL tool in SQL Server 2005) failed. Should I have to truncate the fact table then reload it? I think this is not a better solution?

Any inputs will be appreciated!
bluesky6045Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

regbesCommented:
Hi bluesky6045,

What i do is in my psckage is

Truncate target table
Drop indexes on target table if they exsist
load data into target table
create indexes on target table if they dont exsist

this way whenever you run the package you dont care what is in the target table or wether the previous execution passed or failed

HTH

R.
0
bluesky6045Author Commented:
Hi regbes,

Thank you for your response. Yes, the one you provided  is one of good solutions for this issue. But if I don’t want to truncate the fact table since the window for loading fact table data is just limited (2-3 hours). Is there still a way to load the remaining data to the fact table? Thank you!
0
regbesCommented:
3 other ways

1 to use "stageing" tables
what this means is that you do not load the data directly into you fact table but into a stageing table only once the data you need is fully loaded into the stageing table do you transfer it from the stageing table into your fact table, if your loads are done on a logical breal i.e. date (month) you can use these stageing tables a additional horizontal partitions.

2. make our load process intelegent i.e. check your max id in the fact table then only load data with id's with an id higher than that from your source with this method you need to be sure the data is loaded sequentialy i.e. dont load id 10 before 5 but you gain he benifit of having a robust process that can run almost without careing what happend to the previous load

the third option is touse a hybred of both  
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
bluesky6045Author Commented:
Excellent! I really like the second solution. Thank you so much! I know my question is more related to data warehouse. But we dont have DW channel in EE community. I am quite lucky because I can get you to answer this question. Can I still post any DW (SSIS) related question in SQL Server channel?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server

From novice to tech pro — start learning today.