Solved

Slowly Changing Dimension (Type 3) T-SQL Example

Posted on 2004-08-02
9
1,108 Views
Last Modified: 2012-06-22
Is there a best way to write a T-SQL (SQL Server 2000) so that I can compare the current records with the old records so to perform slowly changing dimension Type3. I am now using a cursor in T-SQL, however it takes ages to fetch exach records and compare the old records. Other suggestion are welcome, e.g. DTS ... etc.
0
Comment
Question by:acwng
  • 2
  • 2
  • 2
  • +1
9 Comments
 
LVL 15

Expert Comment

by:jdlambert1
Comment Utility
I've had cursors speed up by orders of magnitude just by changing READ_ONLY to FAST_FORWARD:

DECLARE Cur1 CURSOR FAST_FORWARD FOR
 <select_statement, , SELECT au_fname FROM pubs.dbo.authors>


0
 
LVL 34

Expert Comment

by:arbert
Comment Utility
Post what you're doing--it sounds like something that could be accomodated with a set operation or possibly a join...
0
 
LVL 1

Author Comment

by:acwng
Comment Utility
Let me post the actual situation:

Current_table - all the current records imported from Source Database

Dimension_table - a history of all records from the Current_table

I declare a cursor to fetch the records in Current_table 1 by 1 and then compare the value of each of the field related to the Dimension_table, if found any record changes between the 2 tables, I will do insert or update in Dimension_table.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 9

Expert Comment

by:xenon_je
Comment Utility
In general cursors are slow, so try to avoid if possible.
In your case to me it sounds as if those statements can be replaced by 2 SQL-statements, an update and an insert

Eg:

update Dimension_table
set ....
from Dimension_table join Current_table on ..
where ... (your condition for which records to update)


select ....into  Dimension_table
from Current_table join Dimension_Table
where ...

Also to not mix up the updated with inserted you could create 2 temp tables in which to put exactly those records from curent table that need to be updated and inserted
EG:
select * into temp_ToInsert from Current_table join Dimension_table on.. where ...
select * into temp_ToUpdate from Current_table join Dimension_table on.. where ...

--now make the update and insert using the Dimension_table and the 2 temp tables created above...

I hope this will give you some ideas...Because the problem was described prety vaguely, the answer could not be more specific...

good luck,
 xenon
0
 
LVL 9

Accepted Solution

by:
xenon_je earned 120 total points
Comment Utility
PS:
the 2 statements
select * into temp_ToInsert from Current_table join Dimension_table on.. where ...
select * into temp_ToUpdate from Current_table join Dimension_table on.. where ...


replace them with:

select * into temp_ToInsert from Current_table where --a condition like field1 not in (select fildX from Dimension_table)
select * into temp_ToUpdate from Current_table  where --a condition like field1 not in (select fildX from Dimension_table)

xenon
0
 
LVL 34

Expert Comment

by:arbert
Comment Utility
Agree with xenon_je and actually, on the update portion, you might find that it's even faster to do a delete and reinsert the entire record--just depends on your data...
0
 
LVL 1

Author Comment

by:acwng
Comment Utility
Thanks all for the support, I'll try out the performance and will let you know the result soon.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Using examples as well as descriptions, and references to Books Online, show the documentation available for datatypes, explain the available data types and show how data can be passed into and out of variables.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now