Solved

SQL Job is running slowly which contains SSIS pacakages

Posted on 2016-10-17
11
51 Views
Last Modified: 2016-10-17
Hi, I have a Datawarehouse  SQL job with 8 steps. Basically the job will populate our Datawarehouse overnight. It will truncate all data in the datawarehouse overnight and reload
the data from the Transactional database.

each step in the job will load each dimention and fact table. Now the problem is its taking around 1 hour to run and I am looking into improve the speed.

I have started looking at the SQL queries in each SSIS package which is used to bring the data from the transacitonal Database. Can ony one please let me know if I need to
check any other areas which might be impacting that sql job  in sql server 2008r2?

Many Thanks
0
Comment
Question by:gvamsimba
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
11 Comments
 
LVL 50

Expert Comment

by:Vitor Montalvão
ID: 41846452
Did you ever think in use SQL Server Analysis Services(SSAS) instead?
0
 

Author Comment

by:gvamsimba
ID: 41846470
Hi Vitor, No, we are only using SSIS for loading our Datawarehouse and then SSRS to generate reports for the business which they then export to Excel.
0
 
LVL 50

Expert Comment

by:Vitor Montalvão
ID: 41846474
I just asked because I think that what you're doing is what SSAS offers. That and much more.
Where are you storing the DW database?
What versions and editions of SQL Server are you working with?
0
Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.

 

Author Comment

by:gvamsimba
ID: 41846477
Hi Vitor, we are using sql server 2008 r2 .

we have a seperate dedicated DB server for our datawarehouse which is different to our transactional DB which is on a different server.
0
 
LVL 50

Expert Comment

by:Vitor Montalvão
ID: 41846485
I guess the DW database doesn't have the Parallel Datawarehouse appliance, right? So basically you're using a transactional database as DW. This can be an acceptable solution if the database is small.
How many queries do you have to store the data from the regular db (source) to the DW db (target)?
Are those queries filtering data or it just load full table from source to target?
0
 

Author Comment

by:gvamsimba
ID: 41846492
There is No parallel DW appliance. our DB is small roughly around 90GB.

to load each fact and dimention table, we have an SSIS package which reads SQL query pointing to the transactional DB without any filters. Its mainly source to target.
0
 
LVL 50

Expert Comment

by:Vitor Montalvão
ID: 41846499
If at the end the DW db will be a copy of the source db then maybe a full restore of the source db into the DW would be a better option. If for some reason if think this is not a valid option then another option is to perform a BULK INSERT of the tables. You don't even need SSIS for that. BCP utility can be used instead.
0
 

Author Comment

by:gvamsimba
ID: 41846502
but how will the full restore work in this case as our DW has a different structure of DB with kimball model ( star flake schema) with Dimention and Fact tables ?
0
 
LVL 50

Accepted Solution

by:
Vitor Montalvão earned 500 total points
ID: 41846505
but how will the full restore work in this case as our DW has a different structure of DB with kimball model ( star flake schema) with Dimention and Fact tables ?
Is what I needed to hear :)
So you must have some transformation task during the data import, right?
My idea is if you're loading all data I guess there is little improvement to be done on the queries and maybe it worth to see other tasks inside the full process.
0
 

Author Comment

by:gvamsimba
ID: 41846600
ok . will do..
0
 

Author Closing Comment

by:gvamsimba
ID: 41846606
ok will do..
0

Featured Post

Salesforce Has Never Been Easier

Improve and reinforce salesforce training & adoption using WalkMe's digital adoption platform. Start saving on costly employee training by creating fast intuitive Walk-Thrus for Salesforce. Claim your Free Account Now

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SQL Server Error: 4060 8 33
How do i delete the last node in an xml in T-SQL 7 29
Section based report in SSRS 14 36
SQL multiple joins 2 21
Slowly Changing Dimension Transformation component in data task flow is very useful for us to manage and control how data changes in SSIS.
I have a large data set and a SSIS package. How can I load this file in multi threading?
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Via a live example combined with referencing Books Online, show some of the information that can be extracted from the Catalog Views in SQL Server.

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question