Link to home
Start Free TrialLog in
Avatar of Aparanjith
Aparanjith

asked on

This query is running for more than 45 minutes, can any one help

This is the query i am running its the query to get the data from different tables to one column which is [Updated bucket], now I have added indexes to the columns whose tables has most number of records. As it is updating every week, probably every day I am not using 'WITH (NOLOCK)'. Can any one tell the best way to reduce the execution time?

,CASE
            --Is nonpar waiver
            WHEN z.SRC_PAR_CD IN ('E','O','S','W')
            THEN 'Nonpar Waiver'
  
      --**Amendment Mailed**
        WHEN z.PROV_TAX_ID IN
                 (SELECT DISTINCT b.PROV_TIN
            FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                 where not exists (select * from dbo.sqs_objector_TINs t where b.PROV_TIN = t.prov_tin))
            THEN 
                  (SELECT DISTINCT b.Mailing 
                  FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                  WHERE z.PROV_TAX_ID = b.PROV_TIN
                  )
		--**Amendment Mailed Wave 3 and 4**
		WHEN z.PROV_TAX_ID In
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Amendment Mailed (3rd Wave)'
				and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
		THEN 'Amendment Mailed (3rd Wave)'
		
		
		 WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Amendment Mailed (4th Wave)'
				and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
		THEN 'Amendment Mailed (4th Wave)' 
		
            --Is Puerto Rico of Lifesynch
            WHEN z.PROV_TAX_ID IN
                  (SELECT DISTINCT a.PROV_TAX_ID 
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.Bucket <> 'Nonpar'
                  )
            THEN 
                  (SELECT DISTINCT a.Bucket
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.PROV_TAX_ID = z.PROV_TAX_ID)

            --**Top Objecting Systems**
            WHEN z.SYSTEMNAME IN 
                  ('ADVENTIST HEALTH SYSTEM','ASCENSION HEALTH ALLIANCE','AULTMAN HEALTH FOUNDATION','BANNER HEALTH SYSTEM','BERT FISH MEDICAL CENTER','BETHESDA MEMORIAL HOSPITAL','BJC HEALTHCARE','BLOUNT MEMORIAL HOSPITAL','BOCA RATON REGIONAL HOSPITAL','CAROMONT HEALTH SYSTEM','CATHOLIC HEALTH INITIATIVES','CATHOLIC HEALTHCARE PARTNERS','CHRISTUS HEALTH',/*'CLEVELAND CLINIC HEALTH SYSTEM',*/'COLUMBUS REGIONAL HEALTHCARE SYSTEM','COMMUNITY HEALTH SYSTEMS, INC','COXHEALTH','HCA','HEALTH MANAGEMENT ASSOCIATES','HUNTSVILLE HOSPITAL HEALTH SYSTEM','INTEGRIS HEALTH','JUPITER MEDICAL CENTER','LEE MEMORIAL HEALTH SYSTEM','MARTIN MEMORIAL HEALTH SYSTEM','MERCY','MT SINAI MEDICAL CENTER (MIAMI)','MUNROE REGIONAL MEDICAL CENTER','NORMAN REGIONAL HEALTH SYSTEM','NORTHSIDE HEALTH SYSTEM','SHANDS HEALTHCARE','SISTERS OF MERCY - SPRINGFIELD, MO','SSM HEALTH CARE','ST LUKES HEALTH SYSTEM','SUMMA HEALTH SYSTEM','SUSQUEHANNA HEALTH SYSTEM','TBD -- TRINITY HEALTH - CATHOLIC HEALTH EAST','UNIVERSITY OF MISSOURI HEALTH SYSTEM','UNIVERSITY OF NEW MEXICO HOSPITALS','UNIVERSITY OF UTAH HEALTH CARE')
            THEN 'Top Objecting Systems' 

            --**Other Objecting Hospitals**
            WHEN (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI') and z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        PACT.dbo.SQS_EDW_Source z
                  WHERE 
                        z.PROV_TAX_ID IN 
                              (SELECT DISTINCT obj.TIN FROM PACT.dbo.SQS_Provider_Tracking obj
                                                WHERE obj.[Objector?] = 'Yes' 
                                         --       AND obj.[TINs Provided By:] <> 'Unknown'--PACT.dbo.SQS_Objectors_01202014b
                              )
                              AND z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                             -- AND z.SEQUEST_AMT < 31000
                            -- AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Other Objecting Hospitals'
       
  --      --**Objecting Physicians**
        WHEN 
                ( z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI') and  z.PROV_TAX_ID IN 
                        (SELECT DISTINCT 
                              z.PROV_TAX_ID
                        FROM 
                              PACT.dbo.SQS_EDW_Source z
                        WHERE 
                              z.PROV_TAX_ID IN 
                                    (SELECT DISTINCT obj.TIN 
                                    FROM PACT.dbo.SQS_Provider_Tracking obj --PACT.dbo.SQS_Objectors_01202014b
                                    WHERE obj.[Objector?] = 'Yes'
                               --     AND obj.[TINs Provided by:] <> 'Unknown'
                                    )
                        AND z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI')
                        ))
            THEN 'Objecting Physicians'   
            
            
--**********ALL OBJECTORS SHOULD HAVE BEEN BUCKETED AT THIS POINT IN THE QUERY**********


--      --**Non-Objecting Hospitals**
            WHEN (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI') and z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        PACT.dbo.SQS_EDW_Source z
                  WHERE 
                        z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                        --AND z.SEQUEST_AMT < 31000
                      --  AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Non-Objecting Hospitals'            
        
  
  
  --		**Outstanding Contracts for Review**
		 WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Non-Objecting Bilateral Physicians')
		Then 'Non-Objecting Bilateral Physicians'
		
			When  z.prov_tax_id in 
			(select distinct 
				p.prov_tax_id
				from dbo.SQS_CoC_Potential_Mail_List p
				where p.amendmentrights <> 'Unilateral')
		THEN 'Non-Objecting Bilateral Physicians'
		
		WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'More Research Needed')
		THEN 'More Research Needed'
		
		WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Objector')
		THEN 'ERROR'
		
       else 'Market Review/Preparing to Mail'
    
    END AS [Updated Bucket]

Open in new window


Thanks,
Ranjit
Avatar of Jerry Miller
Jerry Miller
Flag of United States of America image

You have 8 or 10 select distinct clauses working, that is what is taking so long.  

In the places where you are using SELECT DISTINCT simply to determine if a record exists, you can use SELECT TOP 1. Then once it finds a match, it moves on and doesn't continue to sort the table.

For example in lines 6-11:
WHEN z.PROV_TAX_ID IN
                 (SELECT TOP 1 b.PROV_TIN
            FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                 where not exists (select * from dbo.sqs_objector_TINs t where b.PROV_TIN = t.prov_tin))
            THEN
Avatar of Aparanjith
Aparanjith

ASKER

So you mean, I need to replace every place with TOP 1 where I have Distinct right??
Only in the places where you are using SELECT DISTINCT simply to determine if a record exists. Like in lines 8,18,28,38,54,60,73,94. There are more below those lines, but you get the idea.

When you are using distinct in the 'z.PROV_TAX_ID IN' clause, it looks through the entire table looking for the distinct rows that match. Selecting TOP 1 only returns one row and if it finds ones stops searching.

You may need to leave it in place on rows like 43-46. It really depends on your data on these.

SELECT DISTINCT a.Bucket
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.PROV_TAX_ID = z.PROV_TAX_ID)
I am not able to figure it out, I changed in some places and runned the query, it took more than the usual, can you please specify in the code were to keep them, where not to.
Try it this way:

,CASE
            --Is nonpar waiver
            WHEN z.SRC_PAR_CD IN ('E','O','S','W')
            THEN 'Nonpar Waiver'
 
      --**Amendment Mailed**
        WHEN z.PROV_TAX_ID IN
                 (SELECT TOP 1 b.PROV_TIN
            FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                 where not exists (select * from dbo.sqs_objector_TINs t where b.PROV_TIN = t.prov_tin))
            THEN
                  (SELECT DISTINCT b.Mailing
                  FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                  WHERE z.PROV_TAX_ID = b.PROV_TIN
                  )
            --**Amendment Mailed Wave 3 and 4**
            WHEN z.PROV_TAX_ID In
                  (SELECT TOP 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Amendment Mailed (3rd Wave)'
                        and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
            THEN 'Amendment Mailed (3rd Wave)'
            
            
             WHEN z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Amendment Mailed (4th Wave)'
                        and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
            THEN 'Amendment Mailed (4th Wave)'
            
            --Is Puerto Rico of Lifesynch
            WHEN z.PROV_TAX_ID IN
                  (SELECT TOP 1 a.PROV_TAX_ID
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.Bucket <> 'Nonpar'
                  )
            THEN
                  (SELECT DISTINCT a.Bucket
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.PROV_TAX_ID = z.PROV_TAX_ID)

            --**Top Objecting Systems**
            WHEN z.SYSTEMNAME IN
                  ('ADVENTIST HEALTH SYSTEM','ASCENSION HEALTH ALLIANCE','AULTMAN HEALTH FOUNDATION','BANNER HEALTH SYSTEM','BERT FISH MEDICAL CENTER','BETHESDA MEMORIAL HOSPITAL','BJC HEALTHCARE','BLOUNT MEMORIAL HOSPITAL','BOCA RATON REGIONAL HOSPITAL','CAROMONT HEALTH SYSTEM','CATHOLIC HEALTH INITIATIVES','CATHOLIC HEALTHCARE PARTNERS','CHRISTUS HEALTH',/*'CLEVELAND CLINIC HEALTH SYSTEM',*/'COLUMBUS REGIONAL HEALTHCARE SYSTEM','COMMUNITY HEALTH SYSTEMS, INC','COXHEALTH','HCA','HEALTH MANAGEMENT ASSOCIATES','HUNTSVILLE HOSPITAL HEALTH SYSTEM','INTEGRIS HEALTH','JUPITER MEDICAL CENTER','LEE MEMORIAL HEALTH SYSTEM','MARTIN MEMORIAL HEALTH SYSTEM','MERCY','MT SINAI MEDICAL CENTER (MIAMI)','MUNROE REGIONAL MEDICAL CENTER','NORMAN REGIONAL HEALTH SYSTEM','NORTHSIDE HEALTH SYSTEM','SHANDS HEALTHCARE','SISTERS OF MERCY - SPRINGFIELD, MO','SSM HEALTH CARE','ST LUKES HEALTH SYSTEM','SUMMA HEALTH SYSTEM','SUSQUEHANNA HEALTH SYSTEM','TBD -- TRINITY HEALTH - CATHOLIC HEALTH EAST','UNIVERSITY OF MISSOURI HEALTH SYSTEM','UNIVERSITY OF NEW MEXICO HOSPITALS','UNIVERSITY OF UTAH HEALTH CARE')
            THEN 'Top Objecting Systems'

            --**Other Objecting Hospitals**
            WHEN (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI') and z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        z.PROV_TAX_ID
                  FROM
                        PACT.dbo.SQS_EDW_Source z
                  WHERE
                        z.PROV_TAX_ID IN
                              (SELECT TOP 1 obj.TIN FROM PACT.dbo.SQS_Provider_Tracking obj
                                                WHERE obj.[Objector?] = 'Yes'
                                         --       AND obj.[TINs Provided By:] <> 'Unknown'--PACT.dbo.SQS_Objectors_01202014b
                              )
                              AND z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                             -- AND z.SEQUEST_AMT < 31000
                            -- AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Other Objecting Hospitals'
       
  --      --**Objecting Physicians**
        WHEN
                ( z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI') and  z.PROV_TAX_ID IN
                        (SELECT TOP 1
                              z.PROV_TAX_ID
                        FROM
                              PACT.dbo.SQS_EDW_Source z
                        WHERE
                              z.PROV_TAX_ID IN
                                    (SELECT TOP 1 obj.TIN
                                    FROM PACT.dbo.SQS_Provider_Tracking obj --PACT.dbo.SQS_Objectors_01202014b
                                    WHERE obj.[Objector?] = 'Yes'
                               --     AND obj.[TINs Provided by:] <> 'Unknown'
                                    )
                        AND z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI')
                        ))
            THEN 'Objecting Physicians'  
           
           
--**********ALL OBJECTORS SHOULD HAVE BEEN BUCKETED AT THIS POINT IN THE QUERY**********


--      --**Non-Objecting Hospitals**
            WHEN (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI') and z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        z.PROV_TAX_ID
                  FROM
                        PACT.dbo.SQS_EDW_Source z
                  WHERE
                        z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                        --AND z.SEQUEST_AMT < 31000
                      --  AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Non-Objecting Hospitals'            
       
 
 
  --            **Outstanding Contracts for Review**
             WHEN z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Non-Objecting Bilateral Physicians')
            Then 'Non-Objecting Bilateral Physicians'
            
                  When  z.prov_tax_id in
                  (select TOP 1
                        p.prov_tax_id
                        from dbo.SQS_CoC_Potential_Mail_List p
                        where p.amendmentrights <> 'Unilateral')
            THEN 'Non-Objecting Bilateral Physicians'
            
            WHEN z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'More Research Needed')
            THEN 'More Research Needed'
            
            WHEN z.PROV_TAX_ID IN
                  (SELECT TOP 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Objector')
            THEN 'ERROR'
            
       else 'Market Review/Preparing to Mail'
   
    END AS [Updated Bucket]
SOLUTION
Avatar of Jerry Miller
Jerry Miller
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Looks like a dup to me too. In fact using 'When EXISTS' from the other question would most likely give you a better result than my solution.
No it is not the dup, because, the case statement I am using here and in the other question is slightly different. That is why I have reposted the question. Because in this query there are sub queries within the query, so I am confused where to put the Where exists. Can you please provide the solution with 'WHERE EXISTS'. As I am new to sql server, I am little confused where to put them exactly.
The query is a little different, so the WHEN EXISTS probably doesn't apply since you are searching to see if the PROV_TAX_ID is in  the sub queries.
Exactly, and when I use Select Top 1 or Select 1, it is taking more time than, usual execution. Can you tell me any other solution, as it is not working??
Can anyone respond to this, as we are trying to get the exact values and also and more logic to the query coming days, i need someone who can show me the appropriate way, as its only on a particular column [updated bucket], where its taking time, because when I ran the query removing this column query, it got executed in 3 minutes. Can anyone tell me the way to fix it, as running the query more than an hour is not an appropriate way right? I have two big tables in this query one is dbo.sqs_EDW_source_SMG which has 6 million records and I have added index on Prov_tax_id for it, and another table has 1million records which is [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] and I have added index on Prov_tin column. Do you want me to add any other indexes on the columns, which can run query faster. Please help me out.
What part do you remove that speeds up the query? We should focus there. You can always request attention on a question and hopefully get more people involved.
I mean its a very big code, I have shared only the query which is taking time that is the query which I shared. The query which I shared is for [Update Bucket] column. The rest of the query is for other columns which is running fine. If I add the query which I have shared with you guys which is, [Updated bucket] column, then it is taking longer time.
So all the problem is in the query which I have shared with you.
Do I need to add any indexes or re modify query with something else? Please help me out anyone
Can any one atleast help me in re writing the query into joins, as may be the sub queries taking longer for the query. Can anyone please help me out?
Try commenting sections out and see where the problem lies. I would start by commenting this section since you said that one of the tables is 6 million and the other 1 million records. It has to be in the way you are joining the tables. Have you tried to run an Execution Plan to see what the total cost of the query is?

http://technet.microsoft.com/en-us/library/ms178071%28v=sql.105%29.aspx


--**Amendment Mailed**
        WHEN z.PROV_TAX_ID IN
                 (SELECT 1 b.PROV_TIN
            FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                 where not exists (select * from dbo.sqs_objector_TINs t where b.PROV_TIN = t.prov_tin))
            THEN
                  (SELECT DISTINCT b.Mailing
                  FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                  WHERE z.PROV_TAX_ID = b.PROV_TIN
                  )
            --**Amendment Mailed Wave 3 and 4**
            WHEN z.PROV_TAX_ID In
                  (SELECT 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Amendment Mailed (3rd Wave)'
                        and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
            THEN 'Amendment Mailed (3rd Wave)'
           
           
             WHEN z.PROV_TAX_ID IN
                  (SELECT 1
                        qz.PROV_TIN
                  FROM
                        [PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
                        where qz.Mailing = 'Amendment Mailed (4th Wave)'
                        and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
            THEN 'Amendment Mailed (4th Wave)'
My manager is saying to use the joins instead of sub queries and also suggested to write the indexes on columns which can improve the performance. But Did not know where to go. Struck in between to solve the code.  And coming to the code, the case of "Amendment 3rd and 4th wave itself taking 30 minutes to run".
Avatar of PortletPaul
I like your manager. Make better use of joins (in the from clause).

You are currently using "correlated subqueries" within the select clause and, as a general rule, this is the least efficient way to use subqueries. Such correlated subqueries are often the cause of poor performance; and there are 18 of them, all held in just a single case expression.

However you have chosen to provide us with just that single case expression, I would want to see the entire query before suggesting any improvements.
Sure PortletPaul,
                        Here is the entire code what we are working on, have a look, and we are basically facing performance issue with [updated bucket] column.
USE PACT
Go

--******Create Back-up of SQS_Bucket_Details_SMG and PACT.dbo.SQS_TINtoSystem**********
IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SQS_Bucketed_Details_SMG_Backup]') AND type in (N'U'))
DROP TABLE [dbo].[SQS_Bucketed_Details_SMG_Backup_SMG]
GO
Select *
INTO PACT.dbo.SQS_Bucketed_Details_SMG_Backup_SMG
From PACT.dbo.SQS_Bucketed_Details_SMG_on_SMG

IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SQS_TINtoSystem_Backup]') AND type in (N'U'))
DROP TABLE [dbo].[SQS_TINtoSystem_Backup]
GO
Select * 
INTO PACT.dbo.SQS_TINtoSystem_Backup
from PACT.dbo.SQS_TINtoSystem


--****Create sqs_objector_TINs (Objections and Rejections)****
Drop table pact.dbo.sqs_objector_TINs
select distinct a.TIN as Prov_TIN
Into pact.dbo.sqs_objector_TINs
from pact.dbo.sqs_provider_tracking a
where a.[Objector?] in ('Top Objector','Objector','Rejector')

/*********** Query for SQS_TINtoSyst***********/
DROP TABLE PACT.dbo.SQS_TINtoSystem
select      distinct 
      b.SRC_PROV_ID
      --,case 
      --      when a.SYSTEM_NAME is null 
      --      then 
      --            case 
      --                  when a.CTRCT_GRP_NAME is null 
      --                  then a.PROV_SMG_NAME 
      --                  else a.CTRCT_GRP_NAME 
      --            end 
      --      else a.SYSTEM_NAME 
      --end as SYSTEM_NAME
      ,COALESCE(a.SYSTEM_NAME, a.CTRCT_GRP_NAME, a.PROV_SMG_NAME) AS SYSTEM_NAME
INTO  Pact.Dbo.SQS_TINtoSystem
from 
      PARE.dbo.EDW_PROD_HOSPITAL_MASTER a 
      Inner Join PARE.dbo.EDW_PROD_HOSPITAL_ID_XREF b 
            on a.SMG_ID = b.SMG_ID 
   --   Inner Join Pact.dbo.SQS_EDW_Source q 
			--on b.SRC_PROV_ID = q.PROV_TAX_ID
		where b.SRC_PLATFORM_CD = 'TX' 
      and a.SMG_ID in 
            ( 
            select distinct 
            SMG_ID 
            from PARE.dbo.EDW_PROD_HOSPITAL_ID_XREF 
            where SRC_PLATFORM_CD = 'TX' 
            and SRC_PROV_ID in (select q.PROV_TAX_ID from PACT.dbo.SQS_EDW_Source q) 
            );

/************** Query for SQS_Bucketed_Details_SMG*****************/
DROP TABLE PACT.dbo.SQS_Bucketed_Details_SMG_on_smg
--Create temp table
SELECT z.SYSTEMNAME
	,Z.PROV_TAX_ID
	,z.PROVIDERNAME
	,z.STATECODE
	,z.SRC_PAR_CD
	,z.SEQUEST_AMT
	,Z.LCLM_RSTMT_TREND_CAT_CD
	,Z.SMG_ID
INTO  #SQS_EDW_SOURCE_WithSMG 
FROM PACT.dbo.SQS_EDW_SOURCE_WithSMG z
WHERE YEAR(Z.Incurred_Mth)=2013

-- Create Temp table Q
select distinct 
			x.TIN,
			case when max(x.IND) = 'NYN' 
				then 'Standard' 
            when max(x.IND) = 'YNN' 
				then 'Express' 
            when max(x.IND) = 'NNY' 
				then 'Non_Standard' else 'Mixed' 
           end as INDdesc
Into #Q
FROM
	(SELECT distinct 
				a.tin, 
                MAX(a.express) + MAX(a.StandardInd) + MAX(NonstandardIND) as IND
            from 
				(select r.TIN,
					case when MAX(r.Express) like 'Y%' then 'Y' else 'N' end As Express, 
                    case when MAX(r.Standard) = 'Y' then 'Y' else 'N' end As StandardInd, 
                    case when MAX(r.[Non-Standard]) = 'Y' then 'Y' else 'N' end AS NonstandardIND
                FROM PACT.DBO.SQS_Objectors_01032014 r 
                GROUP BY r.TIN) a
            group by a.TIN) x
        group by x.TIN

--Create Temp table F

Select * 
INTO  #F
FROM(
	 SELECT distinct g.prov_tax_id 
			,g.system_name
			,g.provider
			,case when g.reimburse_mixed = 'Y' then 'Mixed' 
				when g.reimburse_express = 'Y' then 'Express' 
				when g.reimburse_standard = 'Y' then 'Standard' 
				when g.reimburse_NonStandard = 'Y' then 'NonStandard'
			end as IND_Desc
			,g.Time_Period_for_Dispute
			,case when g.Renewal_Date = 'N' and g.Expiration_Date = 'N' 
				then 'Unclear'
			when g.Renewal_Date = 'N' and g.Expiration_Date <> 'N' 
				then 'Termination'
			when g.Renewal_Date <> 'N' and g.Expiration_Date = 'N' 
				then 'Evergreen'
			when g.Renewal_Date <> 'N' and g.Expiration_Date <> 'N' 
				then 'Termination' 
				else 'Unknown' 
			end as 'Renew_Term_Ind'
			,g.Renewal_Date
		FROM 
			(select distinct 
				bb.PROV_TAX_ID1 as prov_tax_id  
				,aa.* 
			from 
				[PACT].[dbo].[Top_600_Hospitals3] aa
				left join pare.dbo.EDW_PROD_HOSPITAL_MASTER bb 
					on --a.CTRCT_GRP_NAME = b.CTRCT_GRP_NAME
					aa.Provider = bb.PROV_SMG_NAME
					--    and (a.SYSTEM_NAME = b.SMG_SYS_NAME or a.SYSTEM_NAME = b.SYSTEM_NAME)
					--and a.ADDR_LINE1 = b.ADDR_LINE1
					and aa.STATE_CD = bb.STATE_CD
					--and a.ZIP_CD = b.ZIP_CD
					and aa.City1 = bb.CITY_NAME                                                                
			where aa.SYSTEM_NAME <> 'SEE ABOVE') g
		where g.system_name <> 'SEE ABOVE') h   
	where h.ind_Desc is not null

---------------------------------Load Dataset into Temp table---------------

SELECT 
	z.SYSTEMNAME
	--,Case when ZXC.[Subsystem Name] <> 'NULL' Then zxc.[SubSystem Name] 
		--else NULL 
		--End AS SubSystemName
		, CASE
		WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT zxc.TIN
			FROM PACT.dbo.SQS_Provider_Tracking zxc
			WHERE zxc.[SubSystem Name] <> 'NULL'
			)
		THEN
			(SELECT DISTINCT [Subsystem Name]
			FROM PACT.dbo.SQS_Provider_Tracking zxc 
			WHERE z.PROV_TAX_ID = zxc.TIN) 
	End As SubSYSTEMNAME
	,z.PROVIDERNAME
	,z.STATECODE
	,z.PROV_TAX_ID
	,z.SRC_PAR_CD
	,SUM(z.SEQUEST_AMT) Actual_Sequestered_Amt
	
	,CASE
            --Is nonpar waiver
            WHEN z.SRC_PAR_CD IN ('E','O','S','W')
            THEN 'Nonpar Waiver'
  
      --**Amendment Mailed**
        WHEN z.PROV_TAX_ID IN
                 (SELECT DISTINCT b.PROV_TIN
            FROM PACT.dbo.SQS_Mailed_TINs_010614 b WITH (NOLOCK )
                 where not exists (select * from dbo.sqs_objector_TINs t where b.PROV_TIN = t.prov_tin))
            THEN 
                  (SELECT DISTINCT b.Mailing 
                  FROM PACT.dbo.SQS_Mailed_TINs_010614 b
                  WHERE z.PROV_TAX_ID = b.PROV_TIN
                  )
		--**Amendment Mailed Wave 3 and 4**
		WHEN z.PROV_TAX_ID In
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Amendment Mailed (3rd Wave)'
				and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
		THEN 'Amendment Mailed (3rd Wave)'
		
		
		 WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Amendment Mailed (4th Wave)'
				and not exists (select * from dbo.sqs_objector_TINs t where qz.PROV_TIN = t.prov_tin))
		THEN 'Amendment Mailed (4th Wave)' 
		
            --Is Puerto Rico of Lifesynch
            WHEN z.PROV_TAX_ID IN
                  (SELECT DISTINCT a.PROV_TAX_ID 
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.Bucket <> 'Nonpar' 
                  )
            THEN 
                  (SELECT DISTINCT a.Bucket
                  FROM PACT.dbo.SQS_NonPar_PR_LS_TINs a
                  WHERE a.PROV_TAX_ID = z.PROV_TAX_ID)

            --**Top Objecting Systems**
            WHEN z.SYSTEMNAME IN 
                  ('ADVENTIST HEALTH SYSTEM','ASCENSION HEALTH ALLIANCE','AULTMAN HEALTH FOUNDATION','BANNER HEALTH SYSTEM','BERT FISH MEDICAL CENTER','BETHESDA MEMORIAL HOSPITAL','BJC HEALTHCARE','SVILLE HOSPITAL HEALTH SYSTEM','INTEGRIS HEALTH','JUPITER MEDICAL CENTER','LEE MEMORIAL ')
            THEN 'Top Objecting Systems' 
            
            WHEN (z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        #SQS_EDW_SOURCE_WithSMG z
                  WHERE 
                        z.PROV_TAX_ID IN 
                              (SELECT DISTINCT obj.TIN FROM PACT.dbo.SQS_Provider_Tracking obj
                                                WHERE obj.[Objector?] = 'Top Objector' 
                                         --       AND obj.[TINs Provided By:] <> 'Unknown'--PACT.dbo.SQS_Objectors_01202014b
                              )
                              AND (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                              OR z.SMG_ID is not null)
                             -- AND z.SEQUEST_AMT < 31000
                            -- AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Top Objecting Systems'
            
            
            --**Other Objecting Hospitals**
            WHEN (z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        #SQS_EDW_SOURCE_WithSMG z
                  WHERE 
                        z.PROV_TAX_ID IN 
                              (SELECT DISTINCT obj.TIN FROM PACT.dbo.SQS_Provider_Tracking obj
                                                WHERE obj.[Objector?] = 'Objector' 
                                         --       AND obj.[TINs Provided By:] <> 'Unknown'--PACT.dbo.SQS_Objectors_01202014b
                              )
                              AND (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                              OR z.SMG_ID is not null)
                             -- AND z.SEQUEST_AMT < 31000
                            -- AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Other Objecting Hospitals'
       
  --      --**Objecting Physicians**
        WHEN 
                (z.PROV_TAX_ID IN 
                        (SELECT DISTINCT 
                              z.PROV_TAX_ID
                        FROM 
                              #SQS_EDW_SOURCE_WithSMG z
                        WHERE 
                              z.PROV_TAX_ID IN 
                                    (SELECT DISTINCT obj.TIN 
                                    FROM PACT.dbo.SQS_Provider_Tracking obj --PACT.dbo.SQS_Objectors_01202014b
                                    WHERE obj.[Objector?] in ('Objector','Top Objector')
                               --     AND obj.[TINs Provided by:] <> 'Unknown'
                                    )
                        AND (z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI') OR z.SMG_ID IS null)
                        ))
            THEN 'Objecting Physicians'   
            
            
            
         --****Rejecting Hospitals****
         WHEN (z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        #SQS_EDW_SOURCE_WithSMG z
                  WHERE 
                        z.PROV_TAX_ID IN 
                              (SELECT DISTINCT obj.TIN FROM PACT.dbo.SQS_Provider_Tracking obj
                                                WHERE obj.[Objector?] = 'Rejector' 
                                         --       AND obj.[TINs Provided By:] <> 'Unknown'--PACT.dbo.SQS_Objectors_01202014b
                              )
                              AND (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI')
                              OR z.SMG_ID is not null)         ))
            THEN 'Rejecting Hospitals'
            
            
          --****Rejecting Physciains****
            WHEN 
                (z.PROV_TAX_ID IN 
                        (SELECT DISTINCT 
                              z.PROV_TAX_ID
                        FROM 
                              #SQS_EDW_SOURCE_WithSMG z
                        WHERE 
                              z.PROV_TAX_ID IN 
                                    (SELECT DISTINCT obj.TIN 
                                    FROM PACT.dbo.SQS_Provider_Tracking obj --PACT.dbo.SQS_Objectors_01202014b
                                    WHERE obj.[Objector?] = 'Rejector'
                               --     AND obj.[TINs Provided by:] <> 'Unknown'
                                    )
                        AND (z.LCLM_RSTMT_TREND_CAT_CD NOT IN ('HO','HI') OR z.SMG_ID IS null)
                        ))
            THEN 'REjecting Physicians'   
            
            
            
--**********ALL OBJECTORS SHOULD HAVE BEEN BUCKETED AT THIS POINT IN THE QUERY**********


--      --**Non-Objecting Hospitals**
            WHEN (z.PROV_TAX_ID IN
                  (SELECT DISTINCT 
                        z.PROV_TAX_ID
                  FROM 
                        #SQS_EDW_SOURCE_WithSMG z
                  WHERE 
                        (z.LCLM_RSTMT_TREND_CAT_CD IN ('HO','HI') OR z.SMG_ID Is not null)
                        --AND z.SEQUEST_AMT < 31000
                      --  AND z.SYSTEMNAME <> 'NULL'
                  ))
            THEN 'Non-Objecting Hospitals'            
        
  
  
  --		**Outstanding Contracts for Review**
		 WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Non-Objecting Bilateral Physicians')
		Then 'Non-Objecting Bilateral Physicians'
		
			When  z.prov_tax_id in 
			(select distinct 
				p.prov_tax_id
				from  dbo.SQS_CoC_Potential_Mail_List p
				where p.amendmentrights <> 'Unilateral')
		THEN 'Non-Objecting Bilateral Physicians'
		
		WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'More Research Needed')
		THEN 'More Research Needed'
		
		WHEN z.PROV_TAX_ID IN
			(SELECT DISTINCT
				qz.PROV_TIN
			FROM
				[PACT].[HUMAD\ARS3766].[SQS_Mailed_TINs] qz
				where qz.Mailing = 'Objector')
		THEN 'ERROR'
		
       else 'Market Review/Preparing to Mail'
    
    END AS [Updated Bucket]


	,COALESCE(q.INDdesc, f.IND_desc) AS INDdesc
    ,f.Time_Period_for_Dispute
    ,f.Renew_Term_Ind
    ,f.Renewal_Date
    ,z.SMG_ID
    ,'' AS OrderedRank
INTO  PACT.DBO.SQS_Bucketed_Details_SMG_on_SMG
From #SQS_EDW_SOURCE_WithSMG z
      
	--left join select * from PACT.dbo.SQS_Provider_Tracking zxc order by zxc.[subsystem name], zxc.tin
	--		ON z.PROV_TAX_ID = zxc.TIN 
			--WHERE zxc.[SubSystem Name] <> 'NULL'
	left join #F f ON f.PROV_TAX_ID = z.PROV_TAX_ID
			AND z.SYSTEMNAME = f.SYSTEM_NAME
			AND z.PROVIDERNAME = f.Provider
	Left join #Q q ON z.PROV_TAX_ID = q.TIN
	GROUP BY z.SYSTEMNAME
		--,Z.[SubsystemName]
		,z.PROVIDERNAME
		,z.STATECODE
		,z.PROV_TAX_ID
		,z.SRC_PAR_CD
		,q.INDdesc
		,f.IND_Desc
		,f.Time_Period_for_Dispute
		,f.Renew_Term_Ind
		,f.Renewal_Date
		,z.SMG_ID
		,z.LCLM_RSTMT_TREND_CAT_CD

/************************** Drop temp tables*********************/
DROP TABLE #SQS_EDW_SOURCE_WithSMG

DROP TABLE #Q

DROP TABLE #F
      /***********************************************************/

Open in new window


This is the entire code we have, we have created some temp tables, and SQS_EDW_SOURCE_WithSMG is the table which has 6 million records and I have created non clustered indexes on Month,Tax_id and SMG_ID on this table. Can you please help me out with the code.?
Need help? Can anyone help me out..

I am attaching the execution plan here, please have a look. And let me know what can i do with it.
Execution-Plan.sqlplan
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I am using indexes on that table, for incurred month, prov tax I'd and smg id
>>"I am using indexes on that table"

Not according to the execution plan, it is doing a table scan - did you look at that image?
Plus, it scans 5.8 million rows and inserts 5.8 million rows, so that where clause (causing the table scan) doesn't seem to achieve much.

Quite frankly this whole query needs so much rework I don't think I can continue with it as I am out this evening and won't have time on the weekend.
Please don't take this the wrong way, but you really need to hire a competent SQL developer to re-factor that whole query.  To put it bluntly it is a mess and will require somewhere between 6 to 8 hours by someone with experience to resolve.
Yes that is what I am trying to do, the place where I work, they do not have the table structure in a proper way, they just dumped the tables from Access to sql server, and they do not have primary keys nor indexes in them. I recently joined here, and they gave me this query to solve, previously it worked for 3 hours now as I added some indexes on columns it went for 40 minutes. But who wants a version of query which is running for 40 minutes. And as I am a person hired for sql they are expecting me do it, but I am unable to do that as I am new to sql and this is my second sql job.
                                                                                       I do not know anyone in other way I could not afford them to hire and pay, that is the reason I have joined as a member in Expert Exchange. It helped me a lot in times where I need it. But need some real help here?????
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks for the valuable comments...
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks paul, and I already removed distinct in places where it does not needed now. And mostly figuring out of how to work on my case logic of [updated bucket] column. As there I am facing problem, in execution plan it is taking only 1 minute to get data into all temp tables, but its taking real time in case logic of update bucket. I am concentrating on that part.
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I've requested that this question be deleted for the following reason:

I did not get the answer. Sorry
I did not get the answer. Sorry
As we told you repeatedly this is not how EE works.  We are volunteers here, we will not do your work for you and a question like this requires about 6-8 hours to resolve adequately.

You will find this site a great resource if you follow the EE Guidelines, failing to do that and you will not get the results you expect.
I think you got plenty of answers and guidance - you weren't too keen on listening to those it seems. Pity. When we work together through these issues you not only get the solution you asked for but learn a great deal more by being involved.
Ranjit,

The sad part about it is that the only one you are hurting is yourself.  The next time you ask a question here, members may think twice about contributing in view of your lack of appreciations and feedback in this question.
PortletPaul and Anthony Perkins are both excellent in SQL I have seen many of their solutions on EE. That is why when they started answering I deferred to them. I am a programmer with a lot of SQL knowledge, but these two are way ahead of me. You should listen to their advice.
Guys, not a problem from your side, it was just a reason i have stated to delete my question. I know that people who contribute here have more than intermediate knowledge, as i am just a rookie I did not got the solution for this. But, I know i have got many of the solutions from EE, and mainly portlet paul and Anthony helped me a lot in my other questions. I am totally convinced, not a problem from your side, I admit.
And one more thing, as the steps told by Paul, such as not to use distinct and to use joins instead of sub queries and many more, i have followed each and every step what you have told to me paul, but the time frame which took like 40 mins before came to 20 mins later after i changed the steps which you have told me. It's not that i am not following what you say, I totally respect to your words what you say, and rely completely on them. It is totally my fault, that being a rookie i opted for heavy loaded job. I agree that you are here to just volunteer us, but not to do my work. With respect to all the members of EE specially Portlet Paul and Anthony, I follow all the steps what they post to my questions. Well thank you for your support guys.