gvamsimba
asked on
SQL Job is running slowly which contains SSIS pacakages
Hi, I have a Datawarehouse SQL job with 8 steps. Basically the job will populate our Datawarehouse overnight. It will truncate all data in the datawarehouse overnight and reload
the data from the Transactional database.
each step in the job will load each dimention and fact table. Now the problem is its taking around 1 hour to run and I am looking into improve the speed.
I have started looking at the SQL queries in each SSIS package which is used to bring the data from the transacitonal Database. Can ony one please let me know if I need to
check any other areas which might be impacting that sql job in sql server 2008r2?
Many Thanks
the data from the Transactional database.
each step in the job will load each dimention and fact table. Now the problem is its taking around 1 hour to run and I am looking into improve the speed.
I have started looking at the SQL queries in each SSIS package which is used to bring the data from the transacitonal Database. Can ony one please let me know if I need to
check any other areas which might be impacting that sql job in sql server 2008r2?
Many Thanks
Did you ever think in use SQL Server Analysis Services(SSAS) instead?
ASKER
Hi Vitor, No, we are only using SSIS for loading our Datawarehouse and then SSRS to generate reports for the business which they then export to Excel.
I just asked because I think that what you're doing is what SSAS offers. That and much more.
Where are you storing the DW database?
What versions and editions of SQL Server are you working with?
Where are you storing the DW database?
What versions and editions of SQL Server are you working with?
ASKER
Hi Vitor, we are using sql server 2008 r2 .
we have a seperate dedicated DB server for our datawarehouse which is different to our transactional DB which is on a different server.
we have a seperate dedicated DB server for our datawarehouse which is different to our transactional DB which is on a different server.
I guess the DW database doesn't have the Parallel Datawarehouse appliance, right? So basically you're using a transactional database as DW. This can be an acceptable solution if the database is small.
How many queries do you have to store the data from the regular db (source) to the DW db (target)?
Are those queries filtering data or it just load full table from source to target?
How many queries do you have to store the data from the regular db (source) to the DW db (target)?
Are those queries filtering data or it just load full table from source to target?
ASKER
There is No parallel DW appliance. our DB is small roughly around 90GB.
to load each fact and dimention table, we have an SSIS package which reads SQL query pointing to the transactional DB without any filters. Its mainly source to target.
to load each fact and dimention table, we have an SSIS package which reads SQL query pointing to the transactional DB without any filters. Its mainly source to target.
If at the end the DW db will be a copy of the source db then maybe a full restore of the source db into the DW would be a better option. If for some reason if think this is not a valid option then another option is to perform a BULK INSERT of the tables. You don't even need SSIS for that. BCP utility can be used instead.
ASKER
but how will the full restore work in this case as our DW has a different structure of DB with kimball model ( star flake schema) with Dimention and Fact tables ?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
ok . will do..
ASKER
ok will do..