?
SolvedPrivate

SSIS Execute SQL Task or Data Flow Task

Posted on 2014-11-13
7
Medium Priority
?
74 Views
Last Modified: 2016-02-11
I have a stored procedure that will pass 3 variables to an SSIS task.  The task should accept the 3 variables/parameters and insert into two tables.  The two tables are fairly large tables which is why I'm using ssis to insert the records.  The count of records range from 1K to 600K to 1 millions records at any given time.

What is the most efficient way to execute the task in SSIS? SSIS Execute SQL Task or Data Flow Task with ADO.Net or OLEDB or ODBC?
0
Comment
Question by:LeVette Alexander
  • 3
  • 3
7 Comments
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439765
Depends - where do you want to do it?

Execute SQL Task hands everything back to the database engine - but you said that you were using SSIS to insert the records, which sort of defeats the object?

The Data Flow task allows SSIS to do it, by loading in the tables, doing it, and then saving the tables - but is that better?

How long does it actually take the database engine to do it? Personally, I don't think a million records, if probably indexed, is over-large.
0
 

Author Comment

by:LeVette Alexander
ID: 40439790
Ordinarily, a millions records isn't much; however we don't have the most optimal database; therefore my thought was to leverage inserting the records using SSIS.

Based on your response, I'm guessing the data task flow makes more sense.  The goal is to insert records into our largest tables quickly and efficiently w/o locking, blocking, etc..

It can take the system up to 30secs to insert.  

Thoughts?
0
 
LVL 24

Expert Comment

by:Phillip Burton
ID: 40439818
Give it a go, and time it.
0
The Firewall Audit Checklist

Preparing for a firewall audit today is almost impossible.
AlgoSec, together with some of the largest global organizations and auditors, has created a checklist to follow when preparing for your firewall audit. Simplify risk mitigation while staying compliant all of the time!

 
LVL 66

Expert Comment

by:Jim Horn
ID: 40439847
>The two tables are fairly large tables which is why I'm using ssis
A case can also be made for partitioning, especially if the data you're entering is for a new month / client / state / whatever from the rest of the data, where a separate partition can be created before this insert.  That would increase the speed of the insert, as you wouldn't have to deal with all of the inserting into table with indexes issues.
0
 

Author Comment

by:LeVette Alexander
ID: 40439926
Jim, partitioning is in the near future.  We are looking for an solution now.

Phillip, so if I understand correctly, using either method is fine.  There is not true gain from one or the other, correct?   I've timed them and the execute sql task is faster.  My goal was to determine if one was better than the other and if so, why?
0
 
LVL 24

Accepted Solution

by:
Phillip Burton earned 2000 total points
ID: 40439935
The execute SQL task may be quicker, because SQL server already has the table, so it doesn't need to load and unload the table.

Trouble is, I don't know enough detail of the tables, indices, and query to understand more fully what you are trying to do, so all I can suggest is to test both out and see what is quicker.
0
 

Author Closing Comment

by:LeVette Alexander
ID: 40440065
Thank you
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My client sends data in an Excel file to me to load them into Staging database. The file contains many sheets that they have same structure. In this article, I would like to share the simple way to load data of multiple sheets by using SSIS.
Here's a requirements document template for an integration project (also known as Extract-Transform-Load or ETL) based on my development experience as an SQL Server Information Services (SSIS) developer over the years.
This tutorial will introduce the viewer to VisualVM for the Java platform application. This video explains an example program and covers the Overview, Monitor, and Heap Dump tabs.
Stellar Phoenix SQL Database Repair software easily fixes the suspect mode issue of SQL Server database. It is a simple process to bring the database from suspect mode to normal mode. Check out the video and fix the SQL database suspect mode problem.

616 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question