[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

SSIS: Pivot with dynamic column names?

Posted on 2010-04-08
5
Medium Priority
?
3,134 Views
Last Modified: 2013-11-10
SSIS 2008 SP1 CU6

I have to write a SSIS package.  There are always a fixed set of columns, and a dynamic set of column value pairs controlled by a table.  This dynamic set of column values can change without notice; this is by design.

I have a SP that issues a query that will return something like this:

userid,useremail,attr_name,seq,attr_value
1,moe@stooges.com,name,Moe Howard
1,moe@stooges.com,hair,mop-top
1,mo3@stooges.com,killer move,eye-poke
2,larry@stooges.com,name,Larry Howard
2,larry@stooges.com,hair,finger in light socket
2,larry@stooges.com,killer move,hammer to forehead
3,curly@stooges.com,name,Curly Howard
3,curly@stooges.com,hair,baldie sour
3,curly@stooges.com,killer move,baseball bat

To pivot into:

userid,email,name,hair,killer move
1,moe@stooges.com,Moe Howard,mop-top, eye poke
2,larry@stooges.com,Larry Howard,finger in light socket,hammer to forehead
3,curly@stooges.com,Curly Howard,baldie sour,baseball bat

a classic pivot...

The problem:  Name, hair and killer move could change to name, number of movies, income bracket and some other demographic with the next execution.  The pivot column name values are *never* static.  There are thousands of variations in just the first database I reviewed.

Is it even possible to do this with a SSIS pivot step in a data flow?  I don't want to waste time, and right now I cannot figure out how to do it.
0
Comment
Question by:johnftamburo
3 Comments
 
LVL 16

Accepted Solution

by:
brad2575 earned 672 total points
ID: 30118869
Just create the SQL query dynamically using variables for the 3 field names.

Something like this:

EXEC('
    select ' + DynamicFieldName1 + ', ' + DynamicFieldName2 + '
    FROM table
')

This does not do all the pivot stuff but you would just fill in the blanks with your variable field names.

If you are using SSIS you can populate 3 variables with the field names you are using, and have a variable for teh SQL select statement you want to use and then use an expression to dynamically build the query variable using the 3 variable field names.
0
 
LVL 16

Assisted Solution

by:Megan Brooks
Megan Brooks earned 664 total points
ID: 30133316
Are you saying that the columns in the input table are dynamic? I don't think SSIS would deal well with that directly, but you still should be able to bring the normalized data into a data flow task pipeline, provided that you do the normalizing in SQL and only present a fixed set of columns to SSIS.
You could do the normalizing, for example, in a stored procedure. It could use the metadata views (INFORMATION_SCHEMA.TABLES, INFORMATION_SCHEMA.COLUMNS, etc.) to identify the dynamic columns, and it could then build the UNION query dynamically and use sp_executesql to execute it.
You could also build the SQL query using a script task and then pass it the query string to the source component using a package variable. In that case you might enclose the script task within a Foreach Loop Container that queries and iterates through the column metadata. The script task would concatenate the query for each column into a query string that could be passed to the script task in a variable. Or the metadata could be read and the query could be built using a Data Flow task, again, leaving the query in a package variable.
0
 
LVL 30

Assisted Solution

by:Reza Rad
Reza Rad earned 664 total points
ID: 30133391
0

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article we will learn how to fix  “Cannot install SQL Server 2014 Service Pack 2: Unable to install windows installer msi file” error ?
A Stored Procedure in Microsoft SQL Server is a powerful feature that it can be used to execute the Data Manipulation Language (DML) or Data Definition Language (DDL). Depending on business requirements, a single Stored Procedure can return differe…
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
Stellar Phoenix SQL Database Repair software easily fixes the suspect mode issue of SQL Server database. It is a simple process to bring the database from suspect mode to normal mode. Check out the video and fix the SQL database suspect mode problem.

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question