I have an SSIS package that is using a Data flow with a SQL command from our ERP system and loading into our Fact table in our datawarehouse. Its a query on an Order Header table with an Inner Join to a Order detail table. This is the source data and it does about 20 Lookups after that and finally a slowly Changing Dimension to see whether or not to Insert or Update into our Fact table. This process took about an hour and 45 mins. So I took the query and loaded it into a SQL table and was gonna use the SQL table as the source for the lookups and then the actual Insert but the Dataflow into the SQL table(230k rows) still takes an hour from this source query. We use Transoft to establish the connection to the source system data and it works with the ADO.NET connection but you cant do a table load you have to use a SQL Command for some reason. I want this to run better and faster so if anyone has any tips let me know. I connected to an ODBC source and was able to see the tables but I never ran it. Please help its very important I get this to run faster...
The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.
Here's a requirements document template for an integration project (also known as Extract-Transform-Load or ETL) based on my development experience as an SQL Server Information Services (SSIS) developer over the years.
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables.
Make a table:
Update a specific column given a specific row using the UPDATE statement:
Remove a set of values using the DELETE s…