Solved

SQL Select

Posted on 2014-09-23
9
257 Views
Last Modified: 2014-09-23
We have a SQL stored procedure that selects and returns records against a SQL DB table containing 3 million + records.

The selection logic is very simple, any record that has a created date greater than a variable that is passed in the procedure. The resulting selected record set can be as few as a couple dozen or number in the 100,000s.

The logic run for each record is extensive, to the extent that it takes a couple of seconds for each record to be processed.

Could performance be improved if we were to do a select * insert into a new table with just the subset of records that we want, and then run the logic against that limited number of records?

Thanks for your help
0
Comment
Question by:ordo
  • 4
  • 2
  • 2
  • +1
9 Comments
 
LVL 39

Expert Comment

by:Kyle Abrahams
Comment Utility
"The logic run for each record is extensive"

Can you expound on that?  What are you doing in the logic run?  Does it make more sense to store the data differently?

Without seeing the relevant sections (or an approximation of) it will be difficult to determine if you're better off using a temporary table.
0
 
LVL 65

Expert Comment

by:Jim Horn
Comment Utility
Mind readers we ain't.  Copy-paste the Stored Procedure into this question in a code block, and perhaps we can help..
0
 

Author Comment

by:ordo
Comment Utility
Kyle,

The logic/operations preformed against each record isn't going to change and is way to involved to bore you with. I guess my question is more related to memory usage and processor utilization.

Thanks
0
 

Author Comment

by:ordo
Comment Utility
To elaborate the stored procedure simple selects about 10 columns from each record that meets the desired created date criteria.

Thanks again.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 69

Assisted Solution

by:ScottPletcher
ScottPletcher earned 334 total points
Comment Utility
With such limited info, the only things that are clear so far are:

1) cluster the table by [created date]
2) do as much set-based processing as you can.  if you must use a cursor, make it as efficient as possible.
3) >> Could performance be improved if we were to do a select * insert into a new table with just the subset of records that we want, and then run the logic against that limited number of records? <<  Not likely, but still possible.
0
 

Author Comment

by:ordo
Comment Utility
Scott,

Table is clustered by [create date] column.

Can you elaborate a little bit on your second suggestion (set-based processing)?

Thanks for your help.
0
 
LVL 69

Accepted Solution

by:
ScottPletcher earned 334 total points
Comment Utility
Rather than doing this:

DECLARE cursor_name CURSOR ... FOR
SELECT ...
FROM ...
...
WHILE loop
    FETCH NEXT FROM ...
    ...
...


as much as possible use standard "SELECT ..." to process the data.


Again, w/o being able to see the code, that's just a very general guideline.
0
 
LVL 39

Assisted Solution

by:Kyle Abrahams
Kyle Abrahams earned 166 total points
Comment Utility
another example:

instead of looping each row and doing

column10 = column8 + column9

he's saying:

update table
set column10 = column8 + column9

The logic may not change, but the way you process the logic can be optimized.
0
 

Author Closing Comment

by:ordo
Comment Utility
Thank you all.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

This article explains how to reset the password of the sa account on a Microsoft SQL Server.  The steps in this article work in SQL 2005, 2008, 2008 R2, 2012, 2014 and 2016.
Load balancing is the method of dividing the total amount of work performed by one computer between two or more computers. Its aim is to get more work done in the same amount of time, ensuring that all the users get served faster.
Using examples as well as descriptions, and references to Books Online, show the different Recovery Models available in SQL Server and explain, as well as show how full, differential and transaction log backups are performed
Viewers will learn how to use the INSERT statement to insert data into their tables. It will also introduce the NULL statement, to show them what happens when no value is giving for any given column.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now