Solved

SSIS package

Posted on 2006-11-14
4
517 Views
Last Modified: 2008-03-06
Hello Expert:

Since SSIS is one of SQL Server 2005's feature, I post my question here. I am sorry if it is not fitting in this topic area.

I have an assignment for creating a SSIS package which loading around 20 million rows data into a fact table in the data mart. My question is: what is the better solution when the package failed duing the loading data period (let say, 5 million rows data has been loaded in the fact table when the SSIS package(ETL tool in SQL Server 2005) failed. Should I have to truncate the fact table then reload it? I think this is not a better solution?

Any inputs will be appreciated!
0
Comment
Question by:bluesky6045
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
4 Comments
 
LVL 11

Expert Comment

by:regbes
ID: 17944849
Hi bluesky6045,

What i do is in my psckage is

Truncate target table
Drop indexes on target table if they exsist
load data into target table
create indexes on target table if they dont exsist

this way whenever you run the package you dont care what is in the target table or wether the previous execution passed or failed

HTH

R.
0
 

Author Comment

by:bluesky6045
ID: 17947596
Hi regbes,

Thank you for your response. Yes, the one you provided  is one of good solutions for this issue. But if I don’t want to truncate the fact table since the window for loading fact table data is just limited (2-3 hours). Is there still a way to load the remaining data to the fact table? Thank you!
0
 
LVL 11

Accepted Solution

by:
regbes earned 200 total points
ID: 17948965
3 other ways

1 to use "stageing" tables
what this means is that you do not load the data directly into you fact table but into a stageing table only once the data you need is fully loaded into the stageing table do you transfer it from the stageing table into your fact table, if your loads are done on a logical breal i.e. date (month) you can use these stageing tables a additional horizontal partitions.

2. make our load process intelegent i.e. check your max id in the fact table then only load data with id's with an id higher than that from your source with this method you need to be sure the data is loaded sequentialy i.e. dont load id 10 before 5 but you gain he benifit of having a robust process that can run almost without careing what happend to the previous load

the third option is touse a hybred of both  
0
 

Author Comment

by:bluesky6045
ID: 17949062
Excellent! I really like the second solution. Thank you so much! I know my question is more related to data warehouse. But we dont have DW channel in EE community. I am quite lucky because I can get you to answer this question. Can I still post any DW (SSIS) related question in SQL Server channel?
0

Featured Post

Visualize your virtual and backup environments

Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Recently we ran in to an issue while running some SQL jobs where we were trying to process the cubes.  We got an error saying failure stating 'NT SERVICE\SQLSERVERAGENT does not have access to Analysis Services. So this is a way to automate that wit…
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question