SolvedPrivate

SSIS Execute SQL Task or Data Flow Task

Posted on 2014-11-13
7
42 Views
Last Modified: 2016-02-11
I have a stored procedure that will pass 3 variables to an SSIS task.  The task should accept the 3 variables/parameters and insert into two tables.  The two tables are fairly large tables which is why I'm using ssis to insert the records.  The count of records range from 1K to 600K to 1 millions records at any given time.

What is the most efficient way to execute the task in SSIS? SSIS Execute SQL Task or Data Flow Task with ADO.Net or OLEDB or ODBC?
0
Comment
Question by:LeVette Alexander
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439765
Depends - where do you want to do it?

Execute SQL Task hands everything back to the database engine - but you said that you were using SSIS to insert the records, which sort of defeats the object?

The Data Flow task allows SSIS to do it, by loading in the tables, doing it, and then saving the tables - but is that better?

How long does it actually take the database engine to do it? Personally, I don't think a million records, if probably indexed, is over-large.
0
 

Author Comment

by:LeVette Alexander
ID: 40439790
Ordinarily, a millions records isn't much; however we don't have the most optimal database; therefore my thought was to leverage inserting the records using SSIS.

Based on your response, I'm guessing the data task flow makes more sense.  The goal is to insert records into our largest tables quickly and efficiently w/o locking, blocking, etc..

It can take the system up to 30secs to insert.  

Thoughts?
0
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439818
Give it a go, and time it.
0
Edgartown IT Case Study

Learn about Edgartown's quest to ensure the safety and security of the entire town's employee and citizen data. Read the case study!

 
LVL 66

Expert Comment

by:Jim Horn
ID: 40439847
>The two tables are fairly large tables which is why I'm using ssis
A case can also be made for partitioning, especially if the data you're entering is for a new month / client / state / whatever from the rest of the data, where a separate partition can be created before this insert.  That would increase the speed of the insert, as you wouldn't have to deal with all of the inserting into table with indexes issues.
0
 

Author Comment

by:LeVette Alexander
ID: 40439926
Jim, partitioning is in the near future.  We are looking for an solution now.

Phillip, so if I understand correctly, using either method is fine.  There is not true gain from one or the other, correct?   I've timed them and the execute sql task is faster.  My goal was to determine if one was better than the other and if so, why?
0
 
LVL 24

Accepted Solution

by:
Phillip Burton earned 500 total points
ID: 40439935
The execute SQL task may be quicker, because SQL server already has the table, so it doesn't need to load and unload the table.

Trouble is, I don't know enough detail of the tables, indices, and query to understand more fully what you are trying to do, so all I can suggest is to test both out and see what is quicker.
0
 

Author Closing Comment

by:LeVette Alexander
ID: 40440065
Thank you
0

Featured Post

Comparison of Amazon Drive, Google Drive, OneDrive

What is Best for Backup: Amazon Drive, Google Drive or MS OneDrive? In this free whitepaper we look at their performance, pricing, and platform availability to help you decide which cloud drive is right for your situation. Download and read the results of our testing for free!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Occasionally there is a need to clean table columns, especially if you have inherited legacy data. There are obviously many ways to accomplish that, including elaborate UPDATE queries with anywhere from one to numerous REPLACE functions (even within…
A couple of weeks ago, my client requested me to implement a SSIS package that allows them to download their files from a FTP server and archives them. Microsoft SSIS is the powerful tool which allows us to proceed multiple files at same time even w…
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question