SolvedPrivate

BI questions

Posted on 2014-02-16
3
22 Views
Last Modified: 2016-02-10
I have few questions data BI development

1. What options are available for ensuring consistent configuration of SSIS packages between development, QA, and production run time environments?

2. The source table for a data flow task has been changed so that one of the varchar(50) columns is now a varchar(100).  I want to know if this will break the data flow task. what steps would I need to take?

3. When designing a high volume data flow pipeline, Is it good to use a sort task? why or why not? What other options are there if I must have sorted data?
0
Comment
Question by:wasabi3689
3 Comments
 
LVL 65

Assisted Solution

by:Jim Horn
Jim Horn earned 150 total points
ID: 39864669
Just so you know, the style in which this question was written makes it look like homework, which is against EE policy.  

1.  Don't know.

2.  If the destination for the column we're talking about is less than 100 characters, it will throw a truncation warning, which may result in an error depeding on the settings.

3.  No.  Sorts are blocking components, which means the entire set must be ready for the sort before it can process, and then the entire set must be sorted before it moves on.  

Using a blocking transaction prevents the package from streaming data.
0
 

Author Comment

by:wasabi3689
ID: 39865399
This is not homework. Homework doesn't have QA, TEST and Production environment. I am researching a BI project and some questions brought up by the developers but I cannot answer.
0
 
LVL 37

Accepted Solution

by:
ValentinoV earned 150 total points
ID: 39870037
In addition to Jim's response:

1. The most common method used is called indirect package configuration.  In short: you set up an OS environment variable that contains the full path to the configuration XML file or a connection string to the configuration database.  This allows you to point to the DEV config on the DEV server, to UAT on the UAT server, and so on...

More details: Best Practices for Integration Services Configurations

2. As long as the length of the data in the source table does not exceed 50 chars I don't think it's a problem.  But once the data gets longer, you'll start getting the truncation issue mentioned by Jim.

3. Why would you want to use the Sort component?  If you need incoming data to be sorted for some reason, do it in the source SELECT statement.

If you really, really want to get your data sorted and can't do it in the source query, work with an intermediary table: do whatever manipulation you need to do to the data and write it, unsorted, to the intermediary table.  Then set up a second flow following the first one that reads from the intermediary table, using an ORDER BY clause in the source query.
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I'm trying, I really am. But I've seen so many wrong approaches involving date(time) boundaries I despair about my inability to explain it. I've seen quite a few recently that define a non-leap year as 364 days, or 366 days and the list goes on. …
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now