Go Premium for a chance to win a PS4. Enter to Win

x
?
SolvedPrivate

BI questions

Posted on 2014-02-16
3
Medium Priority
?
28 Views
Last Modified: 2016-02-10
I have few questions data BI development

1. What options are available for ensuring consistent configuration of SSIS packages between development, QA, and production run time environments?

2. The source table for a data flow task has been changed so that one of the varchar(50) columns is now a varchar(100).  I want to know if this will break the data flow task. what steps would I need to take?

3. When designing a high volume data flow pipeline, Is it good to use a sort task? why or why not? What other options are there if I must have sorted data?
0
Comment
Question by:wasabi3689
3 Comments
 
LVL 66

Assisted Solution

by:Jim Horn
Jim Horn earned 600 total points
ID: 39864669
Just so you know, the style in which this question was written makes it look like homework, which is against EE policy.  

1.  Don't know.

2.  If the destination for the column we're talking about is less than 100 characters, it will throw a truncation warning, which may result in an error depeding on the settings.

3.  No.  Sorts are blocking components, which means the entire set must be ready for the sort before it can process, and then the entire set must be sorted before it moves on.  

Using a blocking transaction prevents the package from streaming data.
0
 

Author Comment

by:wasabi3689
ID: 39865399
This is not homework. Homework doesn't have QA, TEST and Production environment. I am researching a BI project and some questions brought up by the developers but I cannot answer.
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 600 total points
ID: 39870037
In addition to Jim's response:

1. The most common method used is called indirect package configuration.  In short: you set up an OS environment variable that contains the full path to the configuration XML file or a connection string to the configuration database.  This allows you to point to the DEV config on the DEV server, to UAT on the UAT server, and so on...

More details: Best Practices for Integration Services Configurations

2. As long as the length of the data in the source table does not exceed 50 chars I don't think it's a problem.  But once the data gets longer, you'll start getting the truncation issue mentioned by Jim.

3. Why would you want to use the Sort component?  If you need incoming data to be sorted for some reason, do it in the source SELECT statement.

If you really, really want to get your data sorted and can't do it in the source query, work with an intermediary table: do whatever manipulation you need to do to the data and write it, unsorted, to the intermediary table.  Then set up a second flow following the first one that reads from the intermediary table, using an ORDER BY clause in the source query.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why is this different from all of the other step by step guides?  Because I make a living as a DBA and not as a writer and I lived through this experience. Defining the name: When I talk to people they say different names on this subject stuff l…
I'm trying, I really am. But I've seen so many wrong approaches involving date(time) boundaries I despair about my inability to explain it. I've seen quite a few recently that define a non-leap year as 364 days, or 366 days and the list goes on. …
Via a live example, show how to backup a database, simulate a failure backup the tail of the database transaction log and perform the restore.
Via a live example, show how to setup several different housekeeping processes for a SQL Server.

916 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question