SSIS Execute SQL Task or Data Flow Task

I have a stored procedure that will pass 3 variables to an SSIS task.  The task should accept the 3 variables/parameters and insert into two tables.  The two tables are fairly large tables which is why I'm using ssis to insert the records.  The count of records range from 1K to 600K to 1 millions records at any given time.

What is the most efficient way to execute the task in SSIS? SSIS Execute SQL Task or Data Flow Task with ADO.Net or OLEDB or ODBC?
LeVette AlexanderDatabase AdministratorAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Phillip BurtonDirector, Practice Manager and Computing ConsultantCommented:
Depends - where do you want to do it?

Execute SQL Task hands everything back to the database engine - but you said that you were using SSIS to insert the records, which sort of defeats the object?

The Data Flow task allows SSIS to do it, by loading in the tables, doing it, and then saving the tables - but is that better?

How long does it actually take the database engine to do it? Personally, I don't think a million records, if probably indexed, is over-large.
0
LeVette AlexanderDatabase AdministratorAuthor Commented:
Ordinarily, a millions records isn't much; however we don't have the most optimal database; therefore my thought was to leverage inserting the records using SSIS.

Based on your response, I'm guessing the data task flow makes more sense.  The goal is to insert records into our largest tables quickly and efficiently w/o locking, blocking, etc..

It can take the system up to 30secs to insert.  

Thoughts?
0
Phillip BurtonDirector, Practice Manager and Computing ConsultantCommented:
Give it a go, and time it.
0
Firewall Management 201 with Professor Wool

In this whiteboard video, Professor Wool highlights the challenges, benefits and trade-offs of utilizing zero-touch automation for security policy change management. Watch and Learn!

Jim HornMicrosoft SQL Server Developer, Architect, and AuthorCommented:
>The two tables are fairly large tables which is why I'm using ssis
A case can also be made for partitioning, especially if the data you're entering is for a new month / client / state / whatever from the rest of the data, where a separate partition can be created before this insert.  That would increase the speed of the insert, as you wouldn't have to deal with all of the inserting into table with indexes issues.
0
LeVette AlexanderDatabase AdministratorAuthor Commented:
Jim, partitioning is in the near future.  We are looking for an solution now.

Phillip, so if I understand correctly, using either method is fine.  There is not true gain from one or the other, correct?   I've timed them and the execute sql task is faster.  My goal was to determine if one was better than the other and if so, why?
0
Phillip BurtonDirector, Practice Manager and Computing ConsultantCommented:
The execute SQL task may be quicker, because SQL server already has the table, so it doesn't need to load and unload the table.

Trouble is, I don't know enough detail of the tables, indices, and query to understand more fully what you are trying to do, so all I can suggest is to test both out and see what is quicker.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
LeVette AlexanderDatabase AdministratorAuthor Commented:
Thank you
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Development

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.