Link to home
Start Free TrialLog in
Avatar of infotechelg
infotechelgFlag for United States of America

asked on

Need SQL Server Performance Help

I am running SQL Server 2008 and have a question about performance.

I have a sproc that does a simple select on 8 joined tables.The select has some WHERE conditions; many involve “CASE” statements.

All the tables have relationship set up, all the non-PK/FKfields that are used in the conditions have indices set up.

When I execute this proc with most values, it returns thedata set in a second or faster.

With one particular set of values, it takes 4.5-5 seconds toreturn the data set and I cannot figure out why.

I removed conditions and added them back one at a time untilthe problem happened and I found where it was happening. The code listed from lines 71-76 is the offending condition:

ALTER PROCEDURE [dbo].[usp_get_fixtures]
      @SYSTEM_ID INT,
      @FINISH_ID INT= 7,
      @INPUT_ID INT= 3,
      @OUTPUT_ID INT= 1,
      @MOUNT_ID INT= NULL,
      @SUB_SYSTEM2_ID INT= 0,
      @IS_CONFIG BIT= 1
AS

BEGIN

      SET NOCOUNT ON;

      SELECT DISTINCT 
            p.NAME,
            xpis.IS_NEW,
            xpis.IS_LED,
            xpis.IS_CF,
            xpis.IS_ADA,
            xpis.IS_ENERGY_STAR,
            xpis.IS_RAILTONES,
            xpis.PRODUCT_NAME_ID, 
            xfpc.FINISH_CHILD_ID,
            p.SUB_SYSTEM2_ID,
            p.ACCESSORY_REQUIRED,
            p.CATALOG_DESCRIPTIONAS PRODUCT_DESCRIPTION,
            p.CATALOG_DESCRIPTION2AS PRODUCT_DESCRIPTION2,
            p.ELG_SPECSHEETAS SPECSHEET,
            c.COLOR_IDAS DEFAULT_COLOR_ID,
            c.NAMEAS DEFAULT_COLOR
      FROM 
            dbo.xref_SKU_SYSTEMAS xss INNER JOIN
            dbo.tbl_SKUAS sku INNER JOIN
            dbo.tbl_PRODUCT_NAME AS p ON sku.PRODUCT_NAME_ID = p.PRODUCT_NAME_ID ON xss.SKU_ID =sku.SKU_ID LEFT OUTER JOIN
            dbo.tbl_COLORAS c ON p.DEFAULT_COLOR_ID = c.COLOR_ID LEFT OUTER JOIN
            dbo.xref_FINISHES_PARENT_CHILDAS xfpc LEFT OUTER JOIN
            dbo.tbl_FINISHAS f ON xfpc.FINISH_PARENT_ID = f.FINISH_ID ON sku.FINISH_ID = xfpc.FINISH_CHILD_ID LEFT OUTER JOIN
            dbo.xref_PRODUCTNAME_ICONS_SYSTEMSAS xpis ON p.PRODUCT_NAME_ID =xpis.PRODUCT_NAME_ID LEFT OUTER JOIN
            dbo.xref_PRODUCTNAME_MOUNT AS xpm ON p.PRODUCT_NAME_ID =xpm.PRODUCT_NAME_ID

      WHERE 
            (p.SUB_SYSTEM_ID = 9) AND
            (p.IS_OBSOLETE = 0) AND
            (sku.IS_OBSOLETE = 0) AND
            (p.NAME NOT LIKE '%dummy%') AND
            (xss.SYSTEM_ID = @SYSTEM_ID OR xpis.SYSTEM_ID= @SYSTEM_ID) AND
            (
                  (
                        CASE @IS_CONFIG
                              WHEN 0 THEN 0
                              ELSE sku.IS_OPTION
                        END
                  ) = @IS_CONFIG
            ) AND
            (
                  (
                        CASE @SUB_SYSTEM2_ID
                              WHEN 0 THEN 0
                              ELSE p.SUB_SYSTEM2_ID
                        END
                  ) = @SUB_SYSTEM2_ID
            ) AND
            (
                  (
                        CASE @FINISH_ID
                              WHEN 0 THEN 0
                              ELSE sku.FINISH_ID
                        END
                  ) = @FINISH_ID OR
                  (
                        CASE @FINISH_ID
                              WHEN 0 THEN 0
                              ELSE xfpc.FINISH_PARENT_ID
                        END
                  ) = @FINISH_ID
            ) AND
            (
                  (
                        CASE @OUTPUT_ID
                              WHEN 0 THEN 0
                              ELSE sku.OUTPUT_VOLTAGE_ID
                        END
                  ) = @OUTPUT_ID OR sku.OUTPUT_VOLTAGE_ID ISNULL
            ) AND
            (
                  (
                        CASE @INPUT_ID
                              WHEN 0 THEN 0
                              ELSE sku.INPUT_VOLTAGE_ID
                        END
                  ) = @INPUT_ID OR sku.INPUT_VOLTAGE_ID ISNULL

            ) AND
            (
                  (
                        CASE @MOUNT_ID
                              WHEN 0 THEN 0
                              ELSE xpm.MOUNT_ID
                        END
                  ) = @MOUNT_ID OR xpm.MOUNT_ID ISNULL
            )

Open in new window


Note that if I remove the code in red, it runs quickly. Alsonote that even if I remove the code in lines 71-76, and I add the code on line 24, it takes 5-6 seconds to return the data set. So it seems there is something odd with that particular table.

Again, this is only for certain values executed in the stored procedures.

Again, the “xref_FINISHES_PARENT_CHILD” table has relationships and keys set up and is a  simple table with a PK and two FKs. But both FKs point to the same column in the same table.

Here’s the xref_FINISHES_PARENT_CHILD table:

CREATE TABLE [dbo].[xref_FINISHES_PARENT_CHILD](
      [ID] [int] IDENTITY(1,1) NOT NULL,
      [FINISH_PARENT_ID] [int] NULL,
      [FINISH_CHILD_ID] [int] NULL,
 CONSTRAINT[PK_xref_FINISHES_PARENT_CHILD] PRIMARY KEY CLUSTERED 
(
      [ID] ASC

) WITH(PAD_INDEX = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]

) ON [PRIMARY]

GO

EXEC sys.sp_addextendedproperty @name=N'MS_Description', @value=N'The primary key.' ,@level0type=N'SCHEMA',@level0name=N'dbo', @level1type=N'TABLE',@level1name=N'xref_FINISHES_PARENT_CHILD',@level2type=N'COLUMN',@level2name=N'ID'

GO

EXEC sys.sp_addextendedproperty @name=N'MS_Description', @value=N'The foreign key to the finish table. This is the parentfinish.' , @level0type=N'SCHEMA',@level0name=N'dbo', @level1type=N'TABLE',@level1name=N'xref_FINISHES_PARENT_CHILD',@level2type=N'COLUMN',@level2name=N'FINISH_PARENT_ID'

GO

EXEC sys.sp_addextendedproperty @name=N'MS_Description', @value=N'The foreign key to the finish table. This is the childfinish to the parent.' , @level0type=N'SCHEMA',@level0name=N'dbo', @level1type=N'TABLE',@level1name=N'xref_FINISHES_PARENT_CHILD',@level2type=N'COLUMN',@level2name=N'FINISH_CHILD_ID'

GO

EXEC sys.sp_addextendedproperty @name=N'MS_Description', @value=N'A cross reference table that relates finishes to a parentfinish.' , @level0type=N'SCHEMA',@level0name=N'dbo', @level1type=N'TABLE',@level1name=N'xref_FINISHES_PARENT_CHILD'

GO

ALTER TABLE [dbo].[xref_FINISHES_PARENT_CHILD]  WITH CHECK ADD CONSTRAINT [FK_xref_FINISHES_PARENT_CHILD_tbl_FINISH]FOREIGN KEY([FINISH_PARENT_ID])

REFERENCES [dbo].[tbl_FINISH] ([FINISH_ID])

GO

ALTER TABLE [dbo].[xref_FINISHES_PARENT_CHILD]CHECK CONSTRAINT[FK_xref_FINISHES_PARENT_CHILD_tbl_FINISH]

GO

ALTER TABLE [dbo].[xref_FINISHES_PARENT_CHILD]  WITH CHECK ADD  CONSTRAINT[FK_xref_FINISHES_PARENT_CHILD_tbl_FINISH1] FOREIGNKEY([FINISH_CHILD_ID])

REFERENCES [dbo].[tbl_FINISH] ([FINISH_ID])

GO

ALTER TABLE [dbo].[xref_FINISHES_PARENT_CHILD]CHECK CONSTRAINT[FK_xref_FINISHES_PARENT_CHILD_tbl_FINISH1]

GO

Open in new window


Any help would be awesome.

Thanks!
Avatar of Scott Pletcher
Scott Pletcher
Flag of United States of America image

I would try re-coding to (hopefully) allow short-circuiting to be done.
Instead of:
                  (
                        CASE @IS_CONFIG
                              WHEN 0 THEN 0
                              ELSE sku.IS_OPTION
                        END
                  ) = @IS_CONFIG
            )

Try:

                  (
                        @IS_CONFIG = 0 OR @IS_CONFIG = sku.IS_OPTION
                  )

For every such param check.



Then try recompiling the proc every time so that parameter snipping can be used accurately:

ALTER PROC ...
WITH RECOMPILE
AS
...

Typically recomplile time is much less than the time saved during execution.
ASKER CERTIFIED SOLUTION
Avatar of lcohan
lcohan
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of infotechelg

ASKER

lcohan, good point. I'll try reindexing and see if that does it. Then I'll try the other things you suggested.

Scott, thanks for your suggestion. I don't think it's that just because if I remove that one piece of code, it's fine. But, if the indexing thing doesn't work, i'll give it a shot. can't hurt, right?
So, I reindexed all indices in the database using the code pasted below and the problem still exists.

DECLARE @TableName VARCHAR(255)
 DECLARE @sql NVARCHAR(500)
 DECLARE @fillfactor INT
 SET @fillfactor = 80
 DECLARE TableCursor CURSOR FOR
 SELECT OBJECT_SCHEMA_NAME([object_id])+'.'+name AS TableName
 FROM sys.tables
 OPEN TableCursor
 FETCH NEXT FROM TableCursor INTO @TableName
 WHILE @@FETCH_STATUS = 0
 BEGIN
 SET @sql = 'ALTER INDEX ALL ON ' + @TableName + ' REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),@fillfactor) + ')'
 EXEC (@sql)
 FETCH NEXT FROM TableCursor INTO @TableName
 END
 CLOSE TableCursor
 DEALLOCATE TableCursor

Open in new window


Running EXEC sp_updatestats didn't work either.

Regarding the "Execution Plan" I did that but, to be honest, I'm not quite sure what I am looking for.
Absolutely no issue to run that but make sure you use REBUILD with ONLINE = ON on live DB's.

Also, no matter fresh index/stats the issue may still exists for specific data sets and that you could see only from the query plan.

I usually UPDATE STATISTICS on ALL tables involved in a SP and then sp_recompile that SP.
I noticed it suggested I add an index to a different table. I did, but it didn't change anything.
User generated imageOk, so I did the UPDATE STATISTICS, then I added "WITH RECOMPILE" to my proc and executed that.

Then I EXEC the proc again and got the same results.

I took a screenshot of the execution plan where the cost was high, but I can't decipher this.
Sorry - I can't see any code in red and could you please just paste that piece giving the issue if its in the stored proc?

I suspect some (huge) cartesian due to OUTER JOINS where you may need to add another condition to further filter the data but not sure where without being able to see the actual plan. Can you see any thik lines in the execution plan? If you hoover over it should give you a picture of how many rows are produced on the backend record set to support your query
The first hash match by percentage then second are your 100% issue and that is due to the OUTER JOINs
Scott, I tried @IS_CONFIG = 0 OR @IS_CONFIG = sku.IS_OPTION, but that didn't work. Thanks for the suggestion.
>> @IS_CONFIG = 0 OR @IS_CONFIG = sku.IS_OPTION, but that didn't work <<

?  What does "didn't work" mean?

It should do the same thing as the other code, except it avoids the CASE and might allow SQL to use parameter sniffing to treat the expression as a constant rather than re-evaluate it for every row.
Can you add a parrallel index on dbo.tbl_SKUAS like IX_SKU_OBSO but having INCLUDE all columns you use from it in this query? that would be to get rid of the first hash join and clustere index scan xref_SKU_SYSTEM. Similar method can apply for the next hash join.
Just add the index with ONLINE = ON to not lock the table if its online.
lcohan, the code in "red" should have been lines 71-76. My apologies.

I tried getting rid of the outer joins, but that didn't help. Actually made it worse.
Sorry Scott, I meant it didn't fix the problem.
so did you tried adding an index like:

create index IX_SKU_OBSO_COVER on tbl_sku
(same_columns_like_IX_SKU_OBSO_here)
INCLUDE
(PRODUCT_NAME_ID,SKU_ID,FINISH_ID,IS_OPTION,OUTPUT_VOLTAGE_ID,INPUT_VOLTAGE_ID)
with (ONLINE = ON, FILLFACTOR = 80)


This should help to get rid of the hash join and the clustered index scan because all those fields would be now INCLUDed in the new index
What specifically "doesn't work" about this::



ALTER PROCEDURE [dbo].[usp_get_fixtures]
      @SYSTEM_ID INT,
      @FINISH_ID INT= 7,
      @INPUT_ID INT= 3,
      @OUTPUT_ID INT= 1,
      @MOUNT_ID INT= NULL,
      @SUB_SYSTEM2_ID INT= 0,
      @IS_CONFIG BIT= 1
WITH RECOMPILE
AS

BEGIN

      SET NOCOUNT ON;

      SELECT DISTINCT
            p.NAME,
            xpis.IS_NEW,
            xpis.IS_LED,
            xpis.IS_CF,
            xpis.IS_ADA,
            xpis.IS_ENERGY_STAR,
            xpis.IS_RAILTONES,
            xpis.PRODUCT_NAME_ID,
            xfpc.FINISH_CHILD_ID,
            p.SUB_SYSTEM2_ID,
            p.ACCESSORY_REQUIRED,
            p.CATALOG_DESCRIPTIONAS PRODUCT_DESCRIPTION,
            p.CATALOG_DESCRIPTION2AS PRODUCT_DESCRIPTION2,
            p.ELG_SPECSHEETAS SPECSHEET,
            c.COLOR_IDAS DEFAULT_COLOR_ID,
            c.NAMEAS DEFAULT_COLOR
      FROM
            dbo.xref_SKU_SYSTEMAS xss INNER JOIN
            dbo.tbl_SKUAS sku INNER JOIN
            dbo.tbl_PRODUCT_NAME AS p ON sku.PRODUCT_NAME_ID = p.PRODUCT_NAME_ID ON xss.SKU_ID =sku.SKU_ID LEFT OUTER JOIN
            dbo.tbl_COLORAS c ON p.DEFAULT_COLOR_ID = c.COLOR_ID LEFT OUTER JOIN
            dbo.xref_FINISHES_PARENT_CHILDAS xfpc LEFT OUTER JOIN
            dbo.tbl_FINISHAS f ON xfpc.FINISH_PARENT_ID = f.FINISH_ID ON sku.FINISH_ID = xfpc.FINISH_CHILD_ID LEFT OUTER JOIN
            dbo.xref_PRODUCTNAME_ICONS_SYSTEMSAS xpis ON p.PRODUCT_NAME_ID =xpis.PRODUCT_NAME_ID LEFT OUTER JOIN
            dbo.xref_PRODUCTNAME_MOUNT AS xpm ON p.PRODUCT_NAME_ID =xpm.PRODUCT_NAME_ID

      WHERE
            (p.SUB_SYSTEM_ID = 9) AND
            (p.IS_OBSOLETE = 0) AND
            (sku.IS_OBSOLETE = 0) AND
            (p.NAME NOT LIKE '%dummy%') AND
            (xss.SYSTEM_ID = @SYSTEM_ID OR xpis.SYSTEM_ID= @SYSTEM_ID) AND
            (@IS_CONFIG = 0 OR @IS_CONFIG = sku.IS_OPTION) AND
            (@SUB_SYSTEM2_ID = 0 OR @SUB_SYSTEM2_ID = p.SUB_SYSTEM2_ID) AND
            (@FINISH_ID = 0 OR @FINISH_ID = sku.FINISH_ID) AND
            (@FINISH_ID = 0 OR @FINISH_ID = xfpc.FINISH_PARENT_ID) AND
            (@OUTPUT_ID = 0 OR @OUTPUT_ID = sku.OUTPUT_VOLTAGE_ID OR sku.OUTPUT_VOLTAGE_ID IS NULL) AND
            (@INPUT_ID = 0 OR @INPUT_ID = sku.INPUT_VOLTAGE_ID OR sku.INPUT_VOLTAGE_ID IS NULL) AND
            (@MOUNT_ID = 0 OR @MOUNT_ID = xpm.MOUNT_ID OR xpm.MOUNT_ID IS NULL)
           
END
Can you add a parrallel index on dbo.tbl_SKUAS like IX_SKU_OBSO but having INCLUDE all columns you use from it in this query? that would be to get rid of the first hash join and clustere index scan xref_SKU_SYSTEM.

lcohan, i'm sorry, but I'm not following what you're asking me to do.
Scott, I said I meant to say it didn't fix my problem. The query still exceutes slowly.
lcohan, I added this (not sure if that was what I was supposed to do) but it still didn't make the query execute faster.

create index IX_SKU_OBSO_COVER on tbl_sku (IS_OBSOLETE)
INCLUDE (PRODUCT_NAME_ID,SKU_ID,FINISH_ID,IS_OPTION,OUTPUT_VOLTAGE_ID,INPUT_VOLTAGE_ID)
with (ONLINE = ON, FILLFACTOR = 80)

Open in new window


Again, this only happens when I pass certain parameters into the proc. Most parameters I pass return the results set in a second or less.
I understand that and is that when you captured the execution plan?
How does the plan changed with the new index? The first and most expensive hash join (actualy the last on the right end of the picture) should go away.
You need to do a UPDATE STATISTICS tbl_sku and sp_recompile [dbo].[usp_get_fixtures] before you can see a change in plan but causion with sp_recompile in prod environment
It's possible that those specific values hit a "tipping" point, causing SQL to either:

(1) not use an index it previously used;  OR
(2) use an index it previously did not use.

And that is causing it to take additional time to do the query.

If so, outside of query hints, you won't be able to change that.
User generated imagelcohan, ok, I ran the UPDATE STATISTICS and recompiled the proc. Attched is the resulting execution plan.
Scott, yeah, I'm not sure. Do you think I should try deleting the offending table and then recreate it?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Nah, no need for that.

If you have a table that's (badly) fragmented, you should probably reorg or rebuild that table.
Easyest way to "fix" a fragmented table is to eliminate the root cause - recreate the clustered index on it and that will do it. DROP/CREATE with ONLINE = ON should work for you if you know that table is soo badly fragmented.
Also in addition to all of the above from my experience sometimes the data cardinality changes soo much that existing queries and table indexes are no longer sufficient as they are - similar to what you have - just because SQL has to process soo much on the back end with same hardware(RAM) so you may have lots of IO instead of mem utilisation (cached data) for your queries.
and just somewhat side note - you know that <<(p.NAME NOT LIKE '%dummy%')>> will invalidate any index usage on that column even if exists because of the % at both ends of the string. Also you could rewrite just that portion like


(NOT p.NAME LIKE '%dummy%')


and also...I personaly found (perfromance) issues with SQL queries where round brakets ()  are used where not necessary.

I mean you need them in

.....
(xss.SYSTEM_ID = @SYSTEM_ID OR xpis.SYSTEM_ID= @SYSTEM_ID) AND .....

but not necessary in each of those above in the WHERE clause in my opinion.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Not to contradict ScottPletcher however from my experience I found that ALTER INDEX ...REORGANIZE does nothing or not much and I thought the indexes were rebuilt if you look above.

Obviously on a production box you won't drop/create a clustered index just like that in the middle of the day and I think I mentioned that but...again I think is the easiest  way to get rid of fragmentation (and only way to get rid of the unused space) from that table.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Just a thought, can you check what happens if you replace all the parameter values in the sql and execute the select sql instead of the stored procedure.
Check how much time it takes?
I want to thank everyone for their help. While the particular issue wasn't solved by these suggetions, you provided a lot of great DBA-type tips that i'll most definitely use from here on out.

Thanks again.