SolvedPrivate

SSIS Execute SQL Task or Data Flow Task

Posted on 2014-11-13
7
35 Views
Last Modified: 2016-02-11
I have a stored procedure that will pass 3 variables to an SSIS task.  The task should accept the 3 variables/parameters and insert into two tables.  The two tables are fairly large tables which is why I'm using ssis to insert the records.  The count of records range from 1K to 600K to 1 millions records at any given time.

What is the most efficient way to execute the task in SSIS? SSIS Execute SQL Task or Data Flow Task with ADO.Net or OLEDB or ODBC?
0
Comment
Question by:LeVette Alexander
  • 3
  • 3
7 Comments
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439765
Depends - where do you want to do it?

Execute SQL Task hands everything back to the database engine - but you said that you were using SSIS to insert the records, which sort of defeats the object?

The Data Flow task allows SSIS to do it, by loading in the tables, doing it, and then saving the tables - but is that better?

How long does it actually take the database engine to do it? Personally, I don't think a million records, if probably indexed, is over-large.
0
 

Author Comment

by:LeVette Alexander
ID: 40439790
Ordinarily, a millions records isn't much; however we don't have the most optimal database; therefore my thought was to leverage inserting the records using SSIS.

Based on your response, I'm guessing the data task flow makes more sense.  The goal is to insert records into our largest tables quickly and efficiently w/o locking, blocking, etc..

It can take the system up to 30secs to insert.  

Thoughts?
0
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439818
Give it a go, and time it.
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 
LVL 65

Expert Comment

by:Jim Horn
ID: 40439847
>The two tables are fairly large tables which is why I'm using ssis
A case can also be made for partitioning, especially if the data you're entering is for a new month / client / state / whatever from the rest of the data, where a separate partition can be created before this insert.  That would increase the speed of the insert, as you wouldn't have to deal with all of the inserting into table with indexes issues.
0
 

Author Comment

by:LeVette Alexander
ID: 40439926
Jim, partitioning is in the near future.  We are looking for an solution now.

Phillip, so if I understand correctly, using either method is fine.  There is not true gain from one or the other, correct?   I've timed them and the execute sql task is faster.  My goal was to determine if one was better than the other and if so, why?
0
 
LVL 24

Accepted Solution

by:
Phillip Burton earned 500 total points
ID: 40439935
The execute SQL task may be quicker, because SQL server already has the table, so it doesn't need to load and unload the table.

Trouble is, I don't know enough detail of the tables, indices, and query to understand more fully what you are trying to do, so all I can suggest is to test both out and see what is quicker.
0
 

Author Closing Comment

by:LeVette Alexander
ID: 40440065
Thank you
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Determine log file requirements 7 35
all records from previous month 6 45
today minus 2 years SQL server 2008 2 31
Getting max record but maybe not use Group BY 2 17
In this article I will describe the Detach & Attach method as one possible migration process and I will add the extra tasks needed for an upgrade when and where is applied so it will cover all.
Lessons learned during ten years of interviewing for SQL Server Integration Services (SSIS) and other Extract-Transform-Load (ETL) contract roles and two years of staff manager interviewing contractors.
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now