Solved

Need help with TSQL - Pivoting columns using PIVOT or something else

Posted on 2013-06-28
14
322 Views
Last Modified: 2013-06-28
i have data coming from a large query that needs to be transposed. Piviot might not work in this instance, and either way i havent been able to get it to work. This cannot be done in SSRS, it has to be straight from the DB, output as a table for another application to use.

this result is currently generated by dynamic sql, as the USERCURR columns is dynamic based on the calling app. lets say there are columns USERCURR1 through USERCURR10. Not all columns will have data.

The Original table will have some static columns that need to remain, and then instead of the USERCURR columns, they are replaced with two columns, BUCKETNAME and BUCKETAMT. For each row, it get SPLIT into multiple rows based on the number of USERCURR columns. I have attached a spreadsheet which clearly explains the need.

example-for-ee.xlsxexample-for-ee.xlsx

Ideally, if the USERCURR column is null or 0 we would not include it, but thats ok as it could be cleaned up later
0
Comment
Question by:mikegrad7
  • 6
  • 5
  • 3
14 Comments
 
LVL 59

Expert Comment

by:Kevin Cross
ID: 39285137
It sounds like you need the UNPIVOT operator and not the PIVOT one.

Essentially:
UNPIVOT (
    BUCKETAMT
    FOR BUCKETNAME
    IN (USERCURR1, USERCURR2, USERCURR3, USERCURR4, USERCURR5, 
        USERCURR6, USERCURR7, USERCURR8, USERCURR9, USERCURR10)
) upvt

Open in new window


In your dynamic SQL, you could do something like this:
@sql = '...
UNPIVOT (
    BUCKETAMT
    FOR BUCKETNAME
    IN (' + @columns + ')
) upvt
...'

I hope that helps!
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39285207
:) this is an UNpivot, however the code I'm proposing does not use the 'unpivot' feature, instead it uses a different technique which is very efficient to achieve this type of result. It uses a combination of cross apply and 'values'.

The result of this code applied to your sample is:
FUND	INVESTOR OPERATIONNUMBER BUCKETNAME	BUCKETAMT
Fund 1		Inv1	1	USERCURR1	1234
Fund 1		Inv1	1	USERCURR2	237
Fund 1		Inv2	1	USERCURR15	2789
Fund 1		Inv2	1	USERCURR16	1298
Fund 1		Inv3	1	USERCURR2	986
Fund 1		Inv3	1	USERCURR16	432
Fund 1		Inv1	2	USERCURR1	3266
Fund 1		Inv1	2	USERCURR2	674
Fund 1		Inv1	2	USERCURR15	3456
Fund 1		Inv2	2	USERCURR1	6789
Fund 1		Inv2	2	USERCURR15	12567
Fund 1		Inv2	2	USERCURR16	7841

Open in new window

The SQL for this is:
SELECT
      Fund
    , Investor
    , OperationNumber
    , BUCKETNAME
    , BUCKETAMT
FROM original
CROSS APPLY (
	          VALUES
                ('USERCURR1', USERCURR1)
              , ('USERCURR2', USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL

Open in new window

this may be seen/used at http://sqlfiddle.com/#!3/b9a0f/1

there are further references for this technique here. Note this technique is marginally faster than the 'unpivot' (elapsed) and consumes less cpu (results An Alternative (Better?) Method to UNPIVOT (SQL Spackle) By Dwain Camps, 2012/08/02)

As you can see this isn't dynamic code, but it can be made into dynamic.
{+edit copy/paste problem - fixed}
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39285220
oh, I see I have missed a bit, the 'line identifier' - sorry - back soon I hope
0
 
LVL 48

Accepted Solution

by:
PortletPaul earned 500 total points
ID: 39285257
This implememts the 'line number'. Note this is actually based on dense_rank() as you require a repeat of a line number - which this conveniently provides.
FUND	INVESTOR	OPERATIONNUMBER	BUCKETNAME	BUCKETAMT	LINE_NUMBER
Fund 1	Inv1	1	USERCURR1			1234		1
Fund 1	Inv1	1	USERCURR2			237		1
Fund 1	Inv1	2	USERCURR1			3266		2
Fund 1	Inv1	2	USERCURR2			674		2
Fund 1	Inv1	2	USERCURR15			3456		2
Fund 1	Inv2	1	USERCURR15			2789		3
Fund 1	Inv2	1	USERCURR16			1298		3
Fund 1	Inv2	2	USERCURR1			6789		4
Fund 1	Inv2	2	USERCURR15			12567		4
Fund 1	Inv2	2	USERCURR16			7841		4
Fund 1	Inv3	1	USERCURR2			986		5
Fund 1	Inv3	1	USERCURR16			432		5

-- code for the above result:
SELECT
      Fund
    , Investor
    , OperationNumber
    , BUCKETNAME
    , BUCKETAMT
    , dense_rank() over (ORDER BY Fund, Investor, OperationNumber) AS line_number
FROM original
CROSS APPLY (
	          VALUES
                ('USERCURR1',  USERCURR1)
              , ('USERCURR2',  USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL

Open in new window

see http://sqlfiddle.com/#!3/a6d98/1
0
 
LVL 59

Expert Comment

by:Kevin Cross
ID: 39285262
PortletPaul's example reminded me of some key points:
- Ensure the columns have the same data types and length. If not, you may want to have a derived table or common table expression handle CONVERT before you denormalize the data.
- Confirm the value in the empty columns is really NULL and not zero or empty string. If either of the latter two, then you can again use an intermediary query to massage the data before you UNPIVOT.
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39285320
Excellent points, yes there is an inherent assumption your can "stack" these values into a column hence they need to share data type. Good catch mwvisa1.
(do I detect some experience there?)
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39285328
actually, you could also do some tidy-up in the cross apply if needed, e.g.
CROSS APPLY (
                VALUES
                ('USERCURR1',  convert(decimal (18,2),USERCURR1)) -- e.g.
              , ('USERCURR2',  USERCURR2)
              , ('USERCURR15', USERCURR15)
              , ('USERCURR16', USERCURR16)
            ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 1

Author Comment

by:mikegrad7
ID: 39285416
wow, you guys are fast, i need to dig thru these and see what works.

FYI, the line number in the spreadsheet is just for clarity, its not part of the data set.

I am still a bit confused on how i keep the initial columns the same, but ill run some tests.

Thanks
Mike
0
 
LVL 1

Author Comment

by:mikegrad7
ID: 39285422
one other comment, my original table is actualyl the result of some dynamic SQL. Im having trouble getting that into a temp table due to the dynamic SQL issues, and then accessing that table later. I know it adds a bit of a wrinkle. Right now i am going to try and as my dynamic sql as the original table part
0
 
LVL 59

Expert Comment

by:Kevin Cross
ID: 39285433
Ah, do you not have access to edit the original dynamic SQL? The reasoning behind my showing you the UNPIVOT syntax is that you probably already have a variable in the dynamic SQL code with the column names, which you said is variable. Leveraging this columns variable, you can have the dynamic SQL pass you the table already denormalized.

If no, how are you getting the dynamic SQL results? Is it from a stored procedure?
0
 
LVL 1

Author Comment

by:mikegrad7
ID: 39285464
no, i have the dynamic SQL string, i was trying to insert the results to a temp table, inside a stored proc, but a few tests have shown that i can do this whole thing in one step, with the sqyamic sql select statement after the from  (  sql) as original
0
 
LVL 1

Author Closing Comment

by:mikegrad7
ID: 39285760
this help was fantastic. I have things working as expected.
0
 
LVL 1

Author Comment

by:mikegrad7
ID: 39285772
Thanks to everyone and PortletPaul with the best answer! I have this working now.

The dynamic SQL ended up looking something like this
      
	select @sql1 = '
	SELECT
	OPERATION_ID  ,	EFFECTIVE_DATE ,
	APPROVED
FROM 
(	SELECT A.IQID as OPERATION_ID,A.CLOSEDATE as EFFECTIVE_DATE,A.USERBOOL1 as APPROVED,A.DRAFT as DRAFT,' + @bucket_list_string + 'FROM (someplace) ) as original
CROSS APPLY ( VALUES ' + @split_bucket_string + ' ) AS CrossApplied(BUCKETNAME , BUCKETAMT)
WHERE bucketamt IS NOT NULL order by SOMEVALUE'
         

Open in new window


then i just execute(@sql1).
0
 
LVL 48

Expert Comment

by:PortletPaul
ID: 39286038
thanks!, and good work making it dynamic. Cheers, Paul
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Best Query To Insert One Year Calendar 6 44
Remove alpha from alphanumeric 4 58
SQL Query 34 79
Help Parsing a String with SQL Syntax 23 21
If you find yourself in this situation “I have used SELECT DISTINCT but I’m getting duplicates” then I'm sorry to say you are using the wrong SQL technique as it only does one thing which is: produces whole rows that are unique. If the results you a…
This article describes how to use the timestamp of existing data in a database to allow Tableau to calculate the prior work day instead of relying on case statements or if statements to calculate the days of the week.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
With Secure Portal Encryption, the recipient is sent a link to their email address directing them to the email laundry delivery page. From there, the recipient will be required to enter a user name and password to enter the page. Once the recipient …

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now