Solved

Need help with TSQL - Pivoting columns using PIVOT or something else

Posted on 2013-06-28
14
319 Views
Last Modified: 2013-06-28
i have data coming from a large query that needs to be transposed. Piviot might not work in this instance, and either way i havent been able to get it to work. This cannot be done in SSRS, it has to be straight from the DB, output as a table for another application to use.

this result is currently generated by dynamic sql, as the USERCURR columns is dynamic based on the calling app. lets say there are columns USERCURR1 through USERCURR10. Not all columns will have data.

The Original table will have some static columns that need to remain, and then instead of the USERCURR columns, they are replaced with two columns, BUCKETNAME and BUCKETAMT. For each row, it get SPLIT into multiple rows based on the number of USERCURR columns. I have attached a spreadsheet which clearly explains the need.

example-for-ee.xlsxexample-for-ee.xlsx

Ideally, if the USERCURR column is null or 0 we would not include it, but thats ok as it could be cleaned up later
0
Comment
Question by:mikegrad7
  • 6
  • 5
  • 3
14 Comments
 
LVL 59

Expert Comment

by:Kevin Cross
Comment Utility
It sounds like you need the UNPIVOT operator and not the PIVOT one.

Essentially:
UNPIVOT (
    BUCKETAMT
    FOR BUCKETNAME
    IN (USERCURR1, USERCURR2, USERCURR3, USERCURR4, USERCURR5, 
        USERCURR6, USERCURR7, USERCURR8, USERCURR9, USERCURR10)
) upvt

Open in new window


In your dynamic SQL, you could do something like this:
@sql = '...
UNPIVOT (
    BUCKETAMT
    FOR BUCKETNAME
    IN (' + @columns + ')
) upvt
...'

I hope that helps!
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
:) this is an UNpivot, however the code I'm proposing does not use the 'unpivot' feature, instead it uses a different technique which is very efficient to achieve this type of result. It uses a combination of cross apply and 'values'.

The result of this code applied to your sample is:
FUND	INVESTOR OPERATIONNUMBER BUCKETNAME	BUCKETAMT
Fund 1		Inv1	1	USERCURR1	1234
Fund 1		Inv1	1	USERCURR2	237
Fund 1		Inv2	1	USERCURR15	2789
Fund 1		Inv2	1	USERCURR16	1298
Fund 1		Inv3	1	USERCURR2	986
Fund 1		Inv3	1	USERCURR16	432
Fund 1		Inv1	2	USERCURR1	3266
Fund 1		Inv1	2	USERCURR2	674
Fund 1		Inv1	2	USERCURR15	3456
Fund 1		Inv2	2	USERCURR1	6789
Fund 1		Inv2	2	USERCURR15	12567
Fund 1		Inv2	2	USERCURR16	7841

Open in new window

The SQL for this is:
SELECT
      Fund
    , Investor
    , OperationNumber
    , BUCKETNAME
    , BUCKETAMT
FROM original
CROSS APPLY (
	          VALUES
                ('USERCURR1', USERCURR1)
              , ('USERCURR2', USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL

Open in new window

this may be seen/used at http://sqlfiddle.com/#!3/b9a0f/1

there are further references for this technique here. Note this technique is marginally faster than the 'unpivot' (elapsed) and consumes less cpu (results An Alternative (Better?) Method to UNPIVOT (SQL Spackle) By Dwain Camps, 2012/08/02)

As you can see this isn't dynamic code, but it can be made into dynamic.
{+edit copy/paste problem - fixed}
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
oh, I see I have missed a bit, the 'line identifier' - sorry - back soon I hope
0
 
LVL 48

Accepted Solution

by:
PortletPaul earned 500 total points
Comment Utility
This implememts the 'line number'. Note this is actually based on dense_rank() as you require a repeat of a line number - which this conveniently provides.
FUND	INVESTOR	OPERATIONNUMBER	BUCKETNAME	BUCKETAMT	LINE_NUMBER
Fund 1	Inv1	1	USERCURR1			1234		1
Fund 1	Inv1	1	USERCURR2			237		1
Fund 1	Inv1	2	USERCURR1			3266		2
Fund 1	Inv1	2	USERCURR2			674		2
Fund 1	Inv1	2	USERCURR15			3456		2
Fund 1	Inv2	1	USERCURR15			2789		3
Fund 1	Inv2	1	USERCURR16			1298		3
Fund 1	Inv2	2	USERCURR1			6789		4
Fund 1	Inv2	2	USERCURR15			12567		4
Fund 1	Inv2	2	USERCURR16			7841		4
Fund 1	Inv3	1	USERCURR2			986		5
Fund 1	Inv3	1	USERCURR16			432		5

-- code for the above result:
SELECT
      Fund
    , Investor
    , OperationNumber
    , BUCKETNAME
    , BUCKETAMT
    , dense_rank() over (ORDER BY Fund, Investor, OperationNumber) AS line_number
FROM original
CROSS APPLY (
	          VALUES
                ('USERCURR1',  USERCURR1)
              , ('USERCURR2',  USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL

Open in new window

see http://sqlfiddle.com/#!3/a6d98/1
0
 
LVL 59

Expert Comment

by:Kevin Cross
Comment Utility
PortletPaul's example reminded me of some key points:
- Ensure the columns have the same data types and length. If not, you may want to have a derived table or common table expression handle CONVERT before you denormalize the data.
- Confirm the value in the empty columns is really NULL and not zero or empty string. If either of the latter two, then you can again use an intermediary query to massage the data before you UNPIVOT.
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
Excellent points, yes there is an inherent assumption your can "stack" these values into a column hence they need to share data type. Good catch mwvisa1.
(do I detect some experience there?)
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
actually, you could also do some tidy-up in the cross apply if needed, e.g.
CROSS APPLY (
                VALUES
                ('USERCURR1',  convert(decimal (18,2),USERCURR1)) -- e.g.
              , ('USERCURR2',  USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 1

Author Comment

by:mikegrad7
Comment Utility
wow, you guys are fast, i need to dig thru these and see what works.

FYI, the line number in the spreadsheet is just for clarity, its not part of the data set.

I am still a bit confused on how i keep the initial columns the same, but ill run some tests.

Thanks
Mike
0
 
LVL 1

Author Comment

by:mikegrad7
Comment Utility
one other comment, my original table is actualyl the result of some dynamic SQL. Im having trouble getting that into a temp table due to the dynamic SQL issues, and then accessing that table later. I know it adds a bit of a wrinkle. Right now i am going to try and as my dynamic sql as the original table part
0
 
LVL 59

Expert Comment

by:Kevin Cross
Comment Utility
Ah, do you not have access to edit the original dynamic SQL? The reasoning behind my showing you the UNPIVOT syntax is that you probably already have a variable in the dynamic SQL code with the column names, which you said is variable. Leveraging this columns variable, you can have the dynamic SQL pass you the table already denormalized.

If no, how are you getting the dynamic SQL results? Is it from a stored procedure?
0
 
LVL 1

Author Comment

by:mikegrad7
Comment Utility
no, i have the dynamic SQL string, i was trying to insert the results to a temp table, inside a stored proc, but a few tests have shown that i can do this whole thing in one step, with the sqyamic sql select statement after the from  (  sql) as original
0
 
LVL 1

Author Closing Comment

by:mikegrad7
Comment Utility
this help was fantastic. I have things working as expected.
0
 
LVL 1

Author Comment

by:mikegrad7
Comment Utility
Thanks to everyone and PortletPaul with the best answer! I have this working now.

The dynamic SQL ended up looking something like this
      
	select @sql1 = '
	SELECT
	OPERATION_ID  ,	EFFECTIVE_DATE ,
	APPROVED
FROM 
(	SELECT A.IQID as OPERATION_ID,A.CLOSEDATE as EFFECTIVE_DATE,A.USERBOOL1 as APPROVED,A.DRAFT as DRAFT,' + @bucket_list_string + 'FROM (someplace) ) as original
CROSS APPLY ( VALUES ' + @split_bucket_string + ' ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL order by SOMEVALUE'
         

Open in new window


then i just execute(@sql1).
0
 
LVL 48

Expert Comment

by:PortletPaul
Comment Utility
thanks!, and good work making it dynamic. Cheers, Paul
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
SQL 2012 Syntax Error 5 23
SQL query 4 25
SQl query 19 12
SQL Update Query - What's wrong with this. 18 9
Confronted with some SQL you don't know can be a daunting task. It can be even more daunting if that SQL carries some of the old secret codes used in the Ye Olde query syntax, such as: (+)     as used in Oracle;     *=     =*    as used in Sybase …
This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now