SolvedPrivate

SSIS Execute SQL Task or Data Flow Task

Posted on 2014-11-13
7
36 Views
Last Modified: 2016-02-11
I have a stored procedure that will pass 3 variables to an SSIS task.  The task should accept the 3 variables/parameters and insert into two tables.  The two tables are fairly large tables which is why I'm using ssis to insert the records.  The count of records range from 1K to 600K to 1 millions records at any given time.

What is the most efficient way to execute the task in SSIS? SSIS Execute SQL Task or Data Flow Task with ADO.Net or OLEDB or ODBC?
0
Comment
Question by:LeVette Alexander
  • 3
  • 3
7 Comments
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439765
Depends - where do you want to do it?

Execute SQL Task hands everything back to the database engine - but you said that you were using SSIS to insert the records, which sort of defeats the object?

The Data Flow task allows SSIS to do it, by loading in the tables, doing it, and then saving the tables - but is that better?

How long does it actually take the database engine to do it? Personally, I don't think a million records, if probably indexed, is over-large.
0
 

Author Comment

by:LeVette Alexander
ID: 40439790
Ordinarily, a millions records isn't much; however we don't have the most optimal database; therefore my thought was to leverage inserting the records using SSIS.

Based on your response, I'm guessing the data task flow makes more sense.  The goal is to insert records into our largest tables quickly and efficiently w/o locking, blocking, etc..

It can take the system up to 30secs to insert.  

Thoughts?
0
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439818
Give it a go, and time it.
0
Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

 
LVL 65

Expert Comment

by:Jim Horn
ID: 40439847
>The two tables are fairly large tables which is why I'm using ssis
A case can also be made for partitioning, especially if the data you're entering is for a new month / client / state / whatever from the rest of the data, where a separate partition can be created before this insert.  That would increase the speed of the insert, as you wouldn't have to deal with all of the inserting into table with indexes issues.
0
 

Author Comment

by:LeVette Alexander
ID: 40439926
Jim, partitioning is in the near future.  We are looking for an solution now.

Phillip, so if I understand correctly, using either method is fine.  There is not true gain from one or the other, correct?   I've timed them and the execute sql task is faster.  My goal was to determine if one was better than the other and if so, why?
0
 
LVL 24

Accepted Solution

by:
Phillip Burton earned 500 total points
ID: 40439935
The execute SQL task may be quicker, because SQL server already has the table, so it doesn't need to load and unload the table.

Trouble is, I don't know enough detail of the tables, indices, and query to understand more fully what you are trying to do, so all I can suggest is to test both out and see what is quicker.
0
 

Author Closing Comment

by:LeVette Alexander
ID: 40440065
Thank you
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
SQL Pivot Rows To Columns 10 51
SQL Server (2008) Declare Variable (Date) and Set value 6 32
sql help 8 55
Query group by data in SQL Server - cursor? 3 28
If you have heard of RFC822 date formats, they can be quite a challenge in SQL Server. RFC822 is an Internet standard format for email message headers, including all dates within those headers. The RFC822 protocols are available in detail at:   ht…
I have a large data set and a SSIS package. How can I load this file in multi threading?
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.

816 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now