Solved

SSIS package

Posted on 2006-11-14
4
514 Views
Last Modified: 2008-03-06
Hello Expert:

Since SSIS is one of SQL Server 2005's feature, I post my question here. I am sorry if it is not fitting in this topic area.

I have an assignment for creating a SSIS package which loading around 20 million rows data into a fact table in the data mart. My question is: what is the better solution when the package failed duing the loading data period (let say, 5 million rows data has been loaded in the fact table when the SSIS package(ETL tool in SQL Server 2005) failed. Should I have to truncate the fact table then reload it? I think this is not a better solution?

Any inputs will be appreciated!
0
Comment
Question by:bluesky6045
  • 2
  • 2
4 Comments
 
LVL 11

Expert Comment

by:regbes
ID: 17944849
Hi bluesky6045,

What i do is in my psckage is

Truncate target table
Drop indexes on target table if they exsist
load data into target table
create indexes on target table if they dont exsist

this way whenever you run the package you dont care what is in the target table or wether the previous execution passed or failed

HTH

R.
0
 

Author Comment

by:bluesky6045
ID: 17947596
Hi regbes,

Thank you for your response. Yes, the one you provided  is one of good solutions for this issue. But if I don’t want to truncate the fact table since the window for loading fact table data is just limited (2-3 hours). Is there still a way to load the remaining data to the fact table? Thank you!
0
 
LVL 11

Accepted Solution

by:
regbes earned 200 total points
ID: 17948965
3 other ways

1 to use "stageing" tables
what this means is that you do not load the data directly into you fact table but into a stageing table only once the data you need is fully loaded into the stageing table do you transfer it from the stageing table into your fact table, if your loads are done on a logical breal i.e. date (month) you can use these stageing tables a additional horizontal partitions.

2. make our load process intelegent i.e. check your max id in the fact table then only load data with id's with an id higher than that from your source with this method you need to be sure the data is loaded sequentialy i.e. dont load id 10 before 5 but you gain he benifit of having a robust process that can run almost without careing what happend to the previous load

the third option is touse a hybred of both  
0
 

Author Comment

by:bluesky6045
ID: 17949062
Excellent! I really like the second solution. Thank you so much! I know my question is more related to data warehouse. But we dont have DW channel in EE community. I am quite lucky because I can get you to answer this question. Can I still post any DW (SSIS) related question in SQL Server channel?
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Clone table from one server.database to another server.database 24 37
First Max value 3 31
sql server query 18 42
SQL Recursion schedule 13 18
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question