Solved

Take SQL Cursor to Set Base

Posted on 2015-01-30
16
93 Views
Last Modified: 2015-02-05
I have a cursor that goes through a table finds any date with null data, and populates that date with yesterday's values.  The date with null data will only have 1 row, but the previous day may have up to  20 rows. (The date field has a many to many relation).    There may be more than one day in a row  (sequence not table row) that is null (weekends and holidays).  In this case we need to use the last available day and populate the null dates with this data until the next day with data.

Below is how I'm doing it in the cursor.

 
Declare @Table varchar(20) = 'HoldingTable' --This line isn't there normally, I actually
--pass the table name in dynamically; however, for clarity sake, I added this here.

Set @strSQL = 'Select Cast(DayYYYYMMDD as Date) as DayYYYYMMDD
		Into Staging_Cursor
		From ' + @Table +
		' Where FullSymbol is null
		Order by DayYYYYMMDD'
	Exec (@strSQL)


DECLARE CursorDB  Cursor  For
		Select DayYYYYMMDD
		From Staging_Cursor		
		Order by DayYYYYMMDD


	--*****************************************************************
	--Assigns missing days to variables
	Open CursorDB
	Fetch Next From  CursorDB into  @MissingDay
								

	While @@FETCH_STATUS=0
	Begin

		--
		--Get The correct values to insert into missing day
		Set @strSQL_Insert = 
			'Insert Into ' + @Table +
		   ' Select A.DayYYYYMMDD,
			B.FullSymbol 
			,B.TickerSymbol 
			,B.LastPrice
			,B.LastTrade
			,B.CurrentContract
			From ' + @Table + ' A
			Join ' + @Table + ' B ON
				DateAdd(day,-1,''' + @MissingDay + ''') = Cast(B.DayYYYYMMDD as Date)  AND
				''' + @MissingDay + ''' = Cast(A.DayYYYYMMDD as Date)'

			
			Exec sp_executesql @strSQL_Insert

	Fetch Next From  CursorDB into  @MissingDay
	End

	Close CursorDB
	Deallocate CursorDB

Open in new window

0
Comment
Question by:badrhino
  • 11
  • 3
  • 2
16 Comments
 
LVL 50

Expert Comment

by:Lowfatspread
ID: 40580646
ok so what is your problem?

ps.  why are you
1) using a cursor for this (v slow in t-sql)
2) using dynamic sql (insecure)


... background info would help....
what does the system represent?
some example initial data, and the corrected data you would expect...
is this for production use or a one-off...
how much data do you expect to be in the tables...
what is the frequency you expect to run this on...
confirm which version/edition of sql server this is to processed on.
0
 
LVL 69

Assisted Solution

by:ScottPletcher
ScottPletcher earned 500 total points
ID: 40580693
Forget the cursor: unlike for Oracle, cursors are performance destroyers in SQL Server.  Just do something like this instead:


Declare @Table varchar(20) = 'HoldingTable' --This line isn't there normally, I actually
Declare @strSQL_Insert varchar(8000)

Set @strSQL_Insert = REPLACE('
Insert Into $table$
Select Cast(A.DayYYYYMMDD As date) As DayYYYYMMDD
                  ,B.FullSymbol
                  ,B.TickerSymbol
                  ,B.LastPrice
                  ,B.LastTrade
                  ,B.CurrentContract
From $table$ A
Inner Join $table$ B On
     Cast(B.DayYYYYMMDD as Date) = DateAdd(Day, -1, Cast(A.DayYYYYMMDD As date))
Where A.FullSymbol Is Null
', '$table$', ISNULL('[' + PARSENAME(@Table, 2) + '].', 'dbo.') + '[' + PARSENAME(@Table, 1) + ']')

PRINT @strSQL_Insert
--EXEC (@strSQL_Insert)
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580783
LowFat,

My problem is I want to replace the cursor with a set base t-sql.  I'm using dynamic sql because I have to use this multiple times for multiple commodities.  So if it isn't dynamic I have to have at a minimum 3 cursors with the exact same code except a different table name injection isn't really an issue because I'm the only one that will see this.

I'm running sql server 2012, the number of times this will be ran is unknown at this time (at least once), but because it may be ran once a week or more (and for the plain fact I want to try and not use a cursor) I would like to rewrite this.  This is for a production server and the number of rows will grow but as of right now we are looking at only 200,000 observations.

The data is actually cme futures data with each day have x number of contracts for each commodity.  

Data before cursor:

DayYYYYMMDD	FullSymbol	LastPrice	LastTrade	CurrentContract
19960102	FC1996F	60.35	1/2/1996	1
19960102	FC1996H	58.08	1/2/1996	2
19960102	FC1996J	58.13	1/2/1996	3
19960102	FC1996K	58.33	1/2/1996	4
19960102	FC1996Q	60.05	1/2/1996	5
19960102	FC1996U	59.9	1/2/1996	6
19960102	FC1996V	59.75	1/2/1996	7
19960102	FC1996X	60.75	1/2/1996	8
19960103	null	null 	null	null
19960104	FC1996F	59.98	1/4/1996	1
19960104	FC1996H	58.08	1/4/1996	2
19960104	FC1996J	58.05	1/4/1996	3
19960104	FC1996K	58.25	1/4/1996	4
19960104	FC1996Q	59.93	1/4/1996	5
19960104	FC1996U	59.98	1/4/1996	6
19960104	FC1996V	60	1/4/1996	7
19960104	FC1996X	61.05	1/4/1996	8

Open in new window


Data after Cursor:
DayYYYYMMDD	FullSymbol	LastPrice	LastTrade	CurrentContract
19960102	FC1996F	60.35	1/2/1996	1
19960102	FC1996H	58.08	1/2/1996	2
19960102	FC1996J	58.13	1/2/1996	3
19960102	FC1996K	58.33	1/2/1996	4
19960102	FC1996Q	60.05	1/2/1996	5
19960102	FC1996U	59.9	1/2/1996	6
19960102	FC1996V	59.75	1/2/1996	7
19960102	FC1996X	60.75	1/2/1996	8
19960103	FC1996F	60.55	1/3/1996	1
19960103	FC1996H	58.33	1/3/1996	2
19960103	FC1996J	58.35	1/3/1996	3
19960103	FC1996K	58.55	1/3/1996	4
19960103	FC1996Q	60.35	1/3/1996	5
19960103	FC1996U	60.15	1/3/1996	6
19960103	FC1996V	60.18	1/3/1996	7
19960103	FC1996X	61.18	1/3/1996	8
19960104	FC1996F	59.98	1/4/1996	1
19960104	FC1996H	58.08	1/4/1996	2
19960104	FC1996J	58.05	1/4/1996	3
19960104	FC1996K	58.25	1/4/1996	4
19960104	FC1996Q	59.93	1/4/1996	5
19960104	FC1996U	59.98	1/4/1996	6
19960104	FC1996V	60	1/4/1996	7
19960104	FC1996X	61.05	1/4/1996	8

Open in new window

0
 
LVL 50

Expert Comment

by:Lowfatspread
ID: 40580784
what updates fullsymbol... to stop rows always being added for the null condition?
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580787
Guys,

My data after cursor should is a little off.  For all  rows of '19960103' the LastTrade should be '1/2/1996' not '1/3/1996'  Sorry about that.
0
 
LVL 69

Expert Comment

by:ScottPletcher
ID: 40580788
Did you try my non-cursor code?
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580796
Fullsymbol and the other fields are coming from a CME feed.  To get the null rows I'm joiningg the CME table to a numbers table:
	;With Staging1 as (
		Select * 
		  From Contract_Reference_Stage1

		)

		Select *
		From Days A   --Numbers table
		Left Join Staging1 B ON
			Cast(A.DayYYYYMMDD as Date) = B.LastTrade 

Open in new window


I've stripped all the conditions out so we can focus on the issue.  There is a valid reason why I'm using a CTE, but didn't want to clutter the issue.
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580797
Scott,

Not yet.  I had to put a fire out.  I'm trying it right now.
0
VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

 
LVL 1

Author Comment

by:badrhino
ID: 40580808
Scott,

Looking at your code, it appears that we're replacing the table name.  If this is what is happening, I don't think I was clear on what the cursor was being used for (My fault, not very good at explaining things).   The dynamic portion is because I'm running it multiple times for different commodities (thus different tables).  The cursor portion is actually going through each row, finding the days that have null values and then inserting the previous days values for that day.
0
 
LVL 69

Expert Comment

by:ScottPletcher
ID: 40580811
I believe my code does that but in a set-based fashion.  You can do all missing dates in one statement, you don't have to go day by day.
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580820
Great.  Lets dig into then.  I'm having a hard time following it--Why do we use the replace function?
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580830
I think I'm following it now.  Let me run it and see what happens!
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580846
Scott,

Your code semi works.  For instance if there are two day's in a row that have null data it inserts the first day because the previous day had data (Friday).  However, because Sunday's previous day (Saturday) doesn't have any data, it is null still.  This is the problem I was trying to overcome before I went to a cursor.  Any ideas?
0
 
LVL 1

Author Comment

by:badrhino
ID: 40580852
Just a note, we also can't be in the mindset that it will always be two day islands.  For instance, if there is a Holiday it could 3-4 days.
0
 
LVL 1

Accepted Solution

by:
badrhino earned 0 total points
ID: 40581804
Guys,

I ended up combining the two.  Essentially I put the join like Scott suggested in a while loop (Using a while loop) and Insert each set of missing days (e.g. all Saturdays, then on the next iteration all Sundays, etc.).  While still a loop, it is only going to loop at most 4-5 times.  

Thanks for the help!
0
 
LVL 1

Author Closing Comment

by:badrhino
ID: 40590664
I had done Scotts solution before I came to expers-exchange; however, I wouldn't have come to my solution without Scott's input.  That's why I accepted this as the solution, and Scott got the points.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

'Between' is such a common word we rarely think about it but in SQL it has a very specific definition we should be aware of. While most database vendors will have their own unique phrases to describe it (see references at end) the concept in common …
Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
Via a live example, show how to extract insert data into a SQL Server database table using the Import/Export option and Bulk Insert.
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

947 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now