Solved

Very slow sql query using while loop and like select

Posted on 2008-06-17
2
2,964 Views
Last Modified: 2012-05-05
I have the attached SQL query which taking hours to run. It badly needs optimising but I'm no SQL expert!
Any help greatly appreciated!

I think it's the like bit which is slowing the query down as the query strings table is massive.

Thanks,
Tristan
DECLARE @StartTime		DATETIME
DECLARE @EndTime		DATETIME
DECLARE @CatStart		INT
DECLARE @CatEnd			INT
DECLARE @TheDay			DATETIME
DECLARE @TheDayPlusOne	DATETIME
DECLARE @TheCat			INT
DECLARE @Count			INT
DECLARE @Search			VARCHAR(10)
 
SET @StartTime =		'April 17 2008'
SET @EndTime =			'June 17 2008'
SET @CatStart =			1
SET @CatEnd =			11
SET @TheDay =			@startTime
SET @TheDayPlusOne =	        DATEADD(DAY, 1, @TheDay)
SET @TheCat =			@CatStart
 
SET NOCOUNT ON
WHILE @TheDay < @endTime
BEGIN
	WHILE @TheCat < @CatEnd
	BEGIN
		SET @Search =	'%cat=' + CAST(@TheCat AS VARCHAR) + '%'
		SET @Count =	( 
		
		SELECT COUNT (DISTINCT WS.[UNIQUE_IDENTIFIER_ID]) 
		FROM [WAP_SESSIONS] AS WS
		INNER JOIN [PAGE_HITS] AS PH ON WS.[WAP_SESSION_ID] = PH.[WAP_SESSION_ID]
		INNER JOIN [QUERY_STRINGS] AS QS ON PH.[QUERY_STRING_ID] = QS.[QUERY_STRING_ID]
		WHERE QS.[QUERY_STRING] LIKE @Search
		AND WS.[CLIENT_ID] = 50510
		AND WS.[DATE_CREATED] BETWEEN @TheDay AND @TheDayPlusOne
 
		 )
 
		PRINT CAST(@TheCat AS VARCHAR) + ', ' + CAST(@TheDay AS VARCHAR) + ', ' + CAST (@Count AS VARCHAR)
		SET @TheCat =	@TheCat + 1		
	END
	SET @TheCat =		@CatStart
    SET @TheDay =		DATEADD(DAY, 1, @TheDay)
	SET @TheDayPlusOne =DATEADD(DAY, 1, @TheDay)
END
 
SET NOCOUNT OFF

Open in new window

0
Comment
Question by:Tristan Shortland
2 Comments
 
LVL 92

Accepted Solution

by:
Patrick Matthews earned 500 total points
ID: 21803710
Hello TShortland,

For one thing, make sure you have indices set up on the columns used in the joins and in the WHERE
clause.  For another, any time you use the LIKE operator, having an index on that column does you no
good, as the LIKE operator forces a full table scan.

It may be worthwhile to do a first step where you create a temp table with the query strings you want and
then join to that temp table, bypassing the LIKE operator for the main pass...

Regards,

Patrick
0
 
LVL 4

Author Comment

by:Tristan Shortland
ID: 21804554
Hi,
I ended up doing a select where query_string like %cat=% into a temp table and then running the above query using that temp table.
This sped it up massively (from returning 10 rows in 18 minutes to 300 in about 5!).
Thanks for your help,
Tristan
0

Featured Post

NAS Cloud Backup Strategies

This article explains backup scenarios when using network storage. We review the so-called “3-2-1 strategy” and summarize the methods you can use to send NAS data to the cloud

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
sql 2014,  lock limit 5 32
Sql Server group by 10 29
Connecting to multiple databases to create a Dashboard 5 26
SQL Count issue 24 16
This article describes how to use the timestamp of existing data in a database to allow Tableau to calculate the prior work day instead of relying on case statements or if statements to calculate the days of the week.
In this article we will get to know that how can we recover deleted data if it happens accidently. We really can recover deleted rows if we know the time when data is deleted by using the transaction log.
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question