[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 525
  • Last Modified:

SSIS package

Hello Expert:

Since SSIS is one of SQL Server 2005's feature, I post my question here. I am sorry if it is not fitting in this topic area.

I have an assignment for creating a SSIS package which loading around 20 million rows data into a fact table in the data mart. My question is: what is the better solution when the package failed duing the loading data period (let say, 5 million rows data has been loaded in the fact table when the SSIS package(ETL tool in SQL Server 2005) failed. Should I have to truncate the fact table then reload it? I think this is not a better solution?

Any inputs will be appreciated!
0
bluesky6045
Asked:
bluesky6045
  • 2
  • 2
1 Solution
 
regbesCommented:
Hi bluesky6045,

What i do is in my psckage is

Truncate target table
Drop indexes on target table if they exsist
load data into target table
create indexes on target table if they dont exsist

this way whenever you run the package you dont care what is in the target table or wether the previous execution passed or failed

HTH

R.
0
 
bluesky6045Author Commented:
Hi regbes,

Thank you for your response. Yes, the one you provided  is one of good solutions for this issue. But if I don’t want to truncate the fact table since the window for loading fact table data is just limited (2-3 hours). Is there still a way to load the remaining data to the fact table? Thank you!
0
 
regbesCommented:
3 other ways

1 to use "stageing" tables
what this means is that you do not load the data directly into you fact table but into a stageing table only once the data you need is fully loaded into the stageing table do you transfer it from the stageing table into your fact table, if your loads are done on a logical breal i.e. date (month) you can use these stageing tables a additional horizontal partitions.

2. make our load process intelegent i.e. check your max id in the fact table then only load data with id's with an id higher than that from your source with this method you need to be sure the data is loaded sequentialy i.e. dont load id 10 before 5 but you gain he benifit of having a robust process that can run almost without careing what happend to the previous load

the third option is touse a hybred of both  
0
 
bluesky6045Author Commented:
Excellent! I really like the second solution. Thank you so much! I know my question is more related to data warehouse. But we dont have DW channel in EE community. I am quite lucky because I can get you to answer this question. Can I still post any DW (SSIS) related question in SQL Server channel?
0

Featured Post

Receive 1:1 tech help

Solve your biggest tech problems alongside global tech experts with 1:1 help.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now