ExpertHelp79
asked on
Optimize the SQL query
I am sharing a SQL query which is taking 25 minutes to execute
Please help me to optimize
Please help me to optimize
ALTER PROCEDURE [dbo].[unbilledMISReport]
-- Add the parameters for the stored procedure here
@uploaddate as varchar(20)
AS
BEGIN
SET NOCOUNT ON;
declare @sql nvarchar(max)
set @sql =
'Select HAR.PersonNo as PersonNo,HAR.EmployeeName as EmployeeName,HAR.Period as Period,'+
'HAR.ProfitCenter as ProfitCenter,HAR.CostCenter as CostCenter,HAR.ProjectTime as ProjectTime,'+
'YTC.Text as Text,Null as Status,Null as Remark,HAR.Account as Account, HAR.ProjectNumber as ProjectNumber,'+
'CONVERT(VARCHAR(10),HAR.Date, 120) as Date,PC.ProjectTLE as ProjectTLE,EMD.BaseLocation as BaseLocation,'+
'ETES.ManagerName as ManagerName,EMD.ORGManager1 as ORGManager1,HAR.Vertical as Vertical,'+
'ICE.Manual_Auto as Manual_Auto,ICE.GDS_NonGDS as GDS_NonGDS,'+
'convert(varchar(10),GPS.ResourceStartDate, 120) as ResourceStartDate,convert(varchar(10),GPS.ResourceEndDate, 120) as ResourceEndDate,'+
'GPS.Status as Status_Act, GPS.BillingKey as BK,PC.SoldToPt as SoldToPt,AM.AccountDescription as AccountDescription,'+
'ETES.EmployeeEmailID as EmployeeEmail,ETES.ManagerEmailID as EtesManagerEmail,'+
'CONVERT(VARCHAR(10),HAR.UploadDate, 120) As UploadDate '+
'From WeeklyHARReport HAR '+
'left outer Join (select * from (select x.* ,row_number() over (partition by (cast(PersonNo as varchar)+cast(cast(ObjectID as datetime) as varchar)) order by objectid ) as rn from YTCError as x) as y where rn=1) as YTC '+
'on (cast(HAR.PersonNo as varchar) + Cast(cast(HAR.Date as datetime)as varchar)) = '+
'(cast(YTC.PersonNo as varchar) + cast(cast(YTC.ObjectID as datetime) as varchar)) '+
'left outer Join (select * from (select a.* ,row_number() over (partition by ShortIdentification order by ShortIdentification) as rn1 from ProjectConsolidated as a) as b where rn1=1 ) as PC '+
'on HAR.ProjectNumber = PC.ShortIdentification '+
'left outer Join (select * from (select a.* ,row_number() over (partition by EmpID order by EmpID) as rn1 from ETesMgrEmp as a) as b where rn1=1 ) as ETES '+
'on HAR.PersonNo = ETES.EmpID '+
'left outer Join (select * from (select a.* ,row_number() over (partition by EmployeeID order by EmployeeID) as rn1 from EmanageDump as a) as b where rn1=1 ) as EMD '+
'on HAR.PersonNo = EMD.EmployeeID '+
'left outer Join (select * from (select a.* ,row_number() over (partition by SoldToParty order by SoldToParty) as rn1 from ICEntities as a) as b where rn1=1 ) as ICE '+
'on PC.SoldToPt = ICE.SoldToParty '+
'left outer Join (select * from (select a.* ,row_number() over (partition by (WBSCode + EmployeeID) order by EmployeeID) as rn1 from GPSF as a) as b where CONVERT(VARCHAR(10),UploadDate, 120) = '''+ @uploaddate +''') as GPS '+
'on (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) '+
'left outer Join (select * from (select a.* ,row_number() over (partition by Account order by Account) as rn1 from AccountMaster as a) as b where rn1=1 ) as AM '+
'on HAR.Account = AM.Account '+
'Where '+
'CONVERT(VARCHAR(10),HAR.UploadDate, 120) = '''+ @uploaddate +''' and HAR.PersonNo is not null '+
'order by HAR.EmployeeName '
--print @sql
exec sp_executesql @sql,N'@uploaddate varchar(20)',@uploaddate
END
use WITH (NOLOCK) for from tables
It is a bit difficult to optimise the code if you do not know the structure and indexes of all the tables involoved.
First thing to check is that you have an index on the columns referenced in the where clause:
HAR.UploadDate,HAR.PersonN o
Then all the joined tables will also need to be indexed:
YTC.PersonNo, YTC.ObjectID
PC.ShortIdentification
ETES.EmpID
ICE.SoldToParty
GPS.WBSCode, GPS.EmployeeID
AM.Account
Also joining two columns in the = clause of a joined table could be very expensive.
Try the indexes(if they don't exist) and if they do exist then rather change this part of the join query:
on (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) to:
on (HAR.PersonNo = GPS.WBSCode and PC.ProjectTLE = GPS.EmployeeID)
First thing to check is that you have an index on the columns referenced in the where clause:
HAR.UploadDate,HAR.PersonN
Then all the joined tables will also need to be indexed:
YTC.PersonNo, YTC.ObjectID
PC.ShortIdentification
ETES.EmpID
ICE.SoldToParty
GPS.WBSCode, GPS.EmployeeID
AM.Account
Also joining two columns in the = clause of a joined table could be very expensive.
Try the indexes(if they don't exist) and if they do exist then rather change this part of the join query:
on (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) to:
on (HAR.PersonNo = GPS.WBSCode and PC.ProjectTLE = GPS.EmployeeID)
1. don't use sp_executesql... just run the query directly.
2. please explain the relationships between the tables... the row_number() Over is probably expensive
3. please explain why the har.person/har.date and ytc.person/ytc.objectiid;s have to be cast and concatenated?
4. similarly why do (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) need concatenation
5, what does the contents of the @uploaddte parameter look like?
CONVERT(VARCHAR(10),HAR.Up loadDate, 120) = '''+ @uploaddate +''' could probably be better expressed...
the above are just the immediate areas to look at...
the functions and concatenation will have drastically reduced the ability of the query to use any indexes
efficiently... which will be the next area to look at once the actual query has been reformatted.
2. please explain the relationships between the tables... the row_number() Over is probably expensive
3. please explain why the har.person/har.date and ytc.person/ytc.objectiid;s
4. similarly why do (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) need concatenation
5, what does the contents of the @uploaddte parameter look like?
CONVERT(VARCHAR(10),HAR.Up
the above are just the immediate areas to look at...
the functions and concatenation will have drastically reduced the ability of the query to use any indexes
efficiently... which will be the next area to look at once the actual query has been reformatted.
ASKER
1. don't use sp_executesql... just run the query directly.
I wrote as it shows me the exact query
2. please explain the relationships between the tables... the row_number() Over is probably expensive
Yes that is costly but it helps me to pick only the first row of the duplicate entries
3. please explain why the har.person/har.date and ytc.person/ytc.objectiid;s have to be cast and concatenated?
This we are creating a unique code
4. similarly why do (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) need concatenation
This we are creating and comparing a unique code
5, what does the contents of the @uploaddte parameter look like?
CONVERT(VARCHAR(10),HAR.Up loadDate, 120) = '''+ @uploaddate +''' could probably be better expressed...
it gives us yyyy-mm-dd
the above are just the immediate areas to look at...
the functions and concatenation will have drastically reduced the ability of the query to use any indexes
efficiently... which will be the next area to look at once the actual query has been reformatted.
I wrote as it shows me the exact query
2. please explain the relationships between the tables... the row_number() Over is probably expensive
Yes that is costly but it helps me to pick only the first row of the duplicate entries
3. please explain why the har.person/har.date and ytc.person/ytc.objectiid;s
This we are creating a unique code
4. similarly why do (HAR.PersonNo + PC.ProjectTLE) = (GPS.WBSCode + GPS.EmployeeID) need concatenation
This we are creating and comparing a unique code
5, what does the contents of the @uploaddte parameter look like?
CONVERT(VARCHAR(10),HAR.Up
it gives us yyyy-mm-dd
the above are just the immediate areas to look at...
the functions and concatenation will have drastically reduced the ability of the query to use any indexes
efficiently... which will be the next area to look at once the actual query has been reformatted.
we would need the explain plan of the query to see where indexes are missing, eventually.
ASKER
how to make the plan
in the query menu, tick "include actual explain plan"
in SQL management studio there is an icon for show excecution plan
excecutionPlan.PNG
excecutionPlan.PNG
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
There is definitely a problem where clause. Since you are converting the har.UploadDate, SQL Server doesn't know how to use it in an index. Therefore, your query will always do an index scan. It will never be able to do a index seek.
You need to modify your where clause to:
WHERE har.UploadDate = @UploadDate
Attached are some execution plans that validates my above statement. The two execution plans are done on a table that has an index on the DateLogged column. Notice that the only difference in the query is that the second is converting the DateLogged column.
You need to modify your where clause to:
WHERE har.UploadDate = @UploadDate
Attached are some execution plans that validates my above statement. The two execution plans are done on a table that has an index on the DateLogged column. Notice that the only difference in the query is that the second is converting the DateLogged column.
ASKER
some points were effective but not all