How can I improve performance of CTE

Hello , I am processing at least 53 million records and I am using the query provided below, I would like assistance on how I can optimize that query as it is taking days to run

	declare @dateworked datetime 
	set @dateworked = '2014-01-01 00:00:00.000'
	with x as (select sum(m.MinutesWorked) as SumMinutes, datepart(year, dateworked) as Year, COMPANY, m.SickPlan  , Employee 
						from cmk_SickTimeHoldingTransactions  m inner join CMK_HRACCT..cmk_infor_AccrualCalc i on m.SickPlan = i.Sick_plan 
									and DateWorked  > @dateworked 
 								group by COMPANY , Employee, datepart(year, dateworked), m.SickPlan)
select distinct  m.TimeEntryLedgerID, m.Company , m.Employee, c.Sick_plan,x.Year, sum(m.MinutesWorked) as MinutesWorked,  sum(case when m.dateworked > c.EFFECTIVE_DATE and
																					round(x.SumMinutes/60.00,4) > cast(left(isnull(c.MIN_HRS_WRKED,0),3)as int)
																			and c.Sick_plan = m.SickPlan  then round(round(m.MinutesWorked/60.00,4) /c.HRS_ACCRUED_PER_HR,4) 
																		else 0.00	end) as SickHoursAcrued								
	from CMK_HRACCT..cmk_SickTimeHoldingTransactions m with (nolock) inner join x on x.Company = m.Company and x.Employee = m.Employee and x.Year = m.YearWorked
					inner join CMK_HRACCT..cmk_infor_AccrualCalc c on c.Sick_plan = m.SickPlan
					--where m.Employee = 718627
	group by m.Company , m.Employee, c.Sick_plan, x.Year, m.TimeEntryLedgerID

Open in new window

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ste5anSenior DeveloperCommented:
Step 1: Use full qualified object names. Don't use .. as you do in CMK_HRACCT..cmk_infor_AccrualCalc.

Question: all mentioned objects (cmk_SickTimeHoldingTransactions, cmk_infor_AccrualCalc) are tables? Are they indexed to support your query?

Test: I guess, this is a case where a materlized table could be faster:

DECLARE @dateworked DATETIME; 
SET @dateworked = '2014-01-01 00:00:00.000';
        SELECT  SUM(m.MinutesWorked) AS SumMinutes ,
                DATEPART(YEAR, dateworked) AS Year ,
                COMPANY ,
                m.SickPlan ,
        FROM    cmk_SickTimeHoldingTransactions m
                INNER JOIN CMK_HRACCT..cmk_infor_AccrualCalc i ON m.SickPlan = i.Sick_plan
                                                                    AND DateWorked > @dateworked
                Employee ,
                DATEPART(YEAR, dateworked) ,

        m.TimeEntryLedgerID ,
        m.Company ,
        m.Employee ,
        c.Sick_plan ,
        x.Year ,
        SUM(m.MinutesWorked) AS MinutesWorked ,
        SUM(CASE WHEN m.dateworked > c.EFFECTIVE_DATE
                        AND ROUND(x.SumMinutes / 60.00, 4) > CAST(LEFT(ISNULL(c.MIN_HRS_WRKED, 0), 3) AS INT)
                        AND c.Sick_plan = m.SickPlan THEN ROUND(ROUND(m.MinutesWorked / 60.00, 4) / c.HRS_ACCRUED_PER_HR, 4)
                    ELSE 0.00
            END) AS SickHoursAcrued
FROM    CMK_HRACCT..cmk_SickTimeHoldingTransactions m WITH ( NOLOCK )
        INNER JOIN #x x ON x.Company = m.Company
                        AND x.Employee = m.Employee
                        AND x.Year = m.YearWorked
        INNER JOIN CMK_HRACCT..cmk_infor_AccrualCalc c ON c.Sick_plan = m.SickPlan					
GROUP BY m.Company ,
        m.Employee ,
        c.Sick_plan ,
        x.Year ,

Open in new window

And last but not least: What are trying to do? Why two GROUP BY's?

I'm also not sure about your logic. Cause you're filtering the data in the CTE, but you're not in the final query.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Vitor MontalvãoMSSQL Senior EngineerCommented:
You're not using a WHERE clause, meaning that you're returning ALL rows. Is that you pretend to do?
Also, do you have indexes on the foreign keys and on DateWorked column?
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft SQL Server 2008

From novice to tech pro — start learning today.