newknew
asked on
Optimize SQL Query: SQL vs. Access
I'm moving an application from a dedicated Windows Server / IIS to Azure with SQL Database and App Service. The existing application runs on MS Access "server" with Classic ASP pages and it is pretty speedy.
The following query on existing server for example returns all records in about 3 seconds. There are 36,720 records in the "loads" table.
Now that I've moved the DB to Azure SQL the records are returned in about 35 seconds to the Azure App. My current Azure config is Serverless Gen5, 2-8 vCores, 6GB-24GB memory. After increasing the compute power I noticed the results returned faster, so I suspected poor query design.
Question: Can the following query be optimized further? I expected much better performance on SQL Server vs. MS Access "Server"
SELECT dbo.loads.ID, dbo.loads.CustomerID, dbo.loads.CustomerContactI D, dbo.loads.RequestDate, dbo.loads.Equipment, dbo.loads.Commodity, dbo.loads.Stops, dbo.loads.Miles, dbo.loads.CommentsPublic, dbo.loads.GrossQuoted,
dbo.loads.Weight, dbo.loads.Length, dbo.loads.Width, dbo.loads.Height, dbo.loads.CargoValue, dbo.loads.CommentsPrivate, dbo.loads.UserID, dbo.loads.Created, dbo.loads.forfeitload, dbo.loads.voidload, dbo.loads.shipperid,
dbo.loads.shippercontactid , dbo.loads.shippercomments, dbo.loads.consigneeid, dbo.loads.consigneecontact id, dbo.loads.consigneecomment s, dbo.loads.usforwarderid, dbo.loads.usforwarderconta ctid,
dbo.loads.usforwardercomme nts, dbo.loads.billtoid, dbo.loads.billtocontactid, dbo.loads.billtocomments, dbo.loads.mxforwarderid, dbo.loads.mxforwarderconta ctid, dbo.loads.mxforwardercomme nts, dbo.loads.mxcarrierid,
dbo.loads.mxcarriercontact id, dbo.loads.mxcarriercomment s, dbo.loads.mxcarriertrucknu mber, dbo.loads.mxcarriertrailer number, dbo.loads.shipperdate, dbo.loads.shipperdateeta, dbo.loads.shipperdateetaas of,
dbo.loads.shipperdateactua l, dbo.loads.consigneedate, dbo.loads.consigneedateeta , dbo.loads.consigneedateeta asof, dbo.loads.consigneedateact ual, dbo.loads.usforwarderdate, dbo.loads.usforwarderdatee ta,
dbo.loads.usforwarderdatee taasof, dbo.loads.usforwarderdatea ctual, dbo.loads.mxforwarderdate, dbo.loads.mxforwarderdatee ta, dbo.loads.mxforwarderdatee taasof, dbo.loads.mxforwarderdatea ctual, dbo.loads.billingcarrierid ,
dbo.loads.billingcarrierlo adnumber, dbo.loads.bookingcarrierid , dbo.loads.bookingcarrierco ntactid, dbo.loads.bookingcarrierco mments, dbo.loads.bookingcarrierlo adnumber, dbo.loads.haulingcarrierid ,
dbo.loads.haulingcarrierco ntactid, dbo.loads.haulingcarrierco mments, dbo.loads.haulingcarrierlo adnumber, dbo.loads.haulingcarriertr ucknumber, dbo.loads.haulingcarrierdr iver, dbo.loads.haulingcarrierdr ivercontac t,
dbo.loads.haulingcarriertr ailer1, dbo.loads.haulingcarrierda tedispatch , dbo.loads.haulingcarrierfu elcard, dbo.loads.haulingcarrierdr ivercommen ts, dbo.loads.interchangelocat ionid, dbo.loads.interchangelocat iondatein,
dbo.loads.interchangelocat iondateine ta, dbo.loads.interchangelocat iondateine taasof, dbo.loads.interchangelocat iondateina ctual, dbo.loads.interchangelocat iondateout , dbo.loads.interchangelocat iondateout eta,
dbo.loads.interchangelocat iondateout etaasof, dbo.loads.interchangelocat iondateout actual, dbo.loads.interchangelocat ioncomment s, dbo.loads.loadpaid, dbo.loads.loadpaidcomments , dbo.loads.shipperreference number,
dbo.loads.consigneereferen cenumber, dbo.loads.interchangelocat ioncontact id, dbo.loads.bookingcarrierlo adentrycom plete, dbo.loads.haulingcarriertr ailer1prep lan, dbo.loads.loadconfiguratio n, dbo.loads.statustims,
dbo.loads.statusavailablel oads, dbo.loads.statusleadsprepl an, dbo.loads.statusleadsfinal , dbo.loads.statuspaperworky ard, dbo.loads.xloadcomments, dbo.loads.haulingcarriertr ailer2, dbo.loads.statusdispatchre ady,
dbo.loads.consigneedatefin al, dbo.loads.shipperdatefinal , dbo.loads.usforwarderdatef inal, dbo.loads.leadsloadid, dbo.loads.bookingcarrier_s 1, dbo.loads.bookingcarrier_s 2, dbo.loads.bookingcarrier_d 1,
dbo.loads.bookingcarrier_d 2, dbo.loads.BookingCarrierBo okingCode, dbo.loads.BookingCarrierRe ferenceNum ber, dbo.loads.billtoreleased, dbo.loads_totalcharges.tot alcharges, dbo.loads_totalcharges.tot alrevenue,
dbo.locations.Name AS customername, locations_1.Name AS shippername, locations_1.City AS shippercity, locations_1.State AS shipperstate, dbo.states.Country AS shippercountry,
dbo.states.country_mapcode AS shippercountrymapcode, CONCAT_WS(', ', dbo.locations.City, dbo.locations.State) AS shippercitystate, locations_2.Name AS consigneename, locations_2.City AS consigneecity,
locations_2.State AS consigneestate, dbo.states_1.Country AS consigneecountry, dbo.states_1.country_mapco de AS consigneecountrymapcode, CONCAT_WS(', ', locations_2.City, locations_2.State) AS consigneecitystate,
locations_3.Name AS haulingcarriername, dbo.locations4.Name AS billtoname, dbo.bookingcodes.BookingCo deName
FROM dbo.loads LEFT OUTER JOIN
dbo.locations ON dbo.loads.CustomerID = dbo.locations.ID LEFT OUTER JOIN
dbo.locations4 ON dbo.loads.billtoid = dbo.locations4.ID LEFT OUTER JOIN
dbo.locations3 AS locations_3 ON dbo.loads.haulingcarrierid = locations_3.ID LEFT OUTER JOIN
dbo.loads_totalcharges ON dbo.loads.ID = dbo.loads_totalcharges.Loa dID LEFT OUTER JOIN
dbo.states_1 RIGHT OUTER JOIN
dbo.locations2 AS locations_2 ON dbo.states_1.Code = locations_2.State ON dbo.loads.consigneeid = locations_2.ID LEFT OUTER JOIN
dbo.states RIGHT OUTER JOIN
dbo.locations1 AS locations_1 ON dbo.states.Code = locations_1.State ON dbo.loads.shipperid = locations_1.ID LEFT OUTER JOIN
dbo.bookingcodes ON dbo.loads.BookingCarrierBo okingCode = dbo.bookingcodes.BookingCo deID
The following query on existing server for example returns all records in about 3 seconds. There are 36,720 records in the "loads" table.
Now that I've moved the DB to Azure SQL the records are returned in about 35 seconds to the Azure App. My current Azure config is Serverless Gen5, 2-8 vCores, 6GB-24GB memory. After increasing the compute power I noticed the results returned faster, so I suspected poor query design.
Question: Can the following query be optimized further? I expected much better performance on SQL Server vs. MS Access "Server"
SELECT dbo.loads.ID, dbo.loads.CustomerID, dbo.loads.CustomerContactI
dbo.loads.Weight, dbo.loads.Length, dbo.loads.Width, dbo.loads.Height, dbo.loads.CargoValue, dbo.loads.CommentsPrivate,
dbo.loads.shippercontactid
dbo.loads.usforwardercomme
dbo.loads.mxcarriercontact
dbo.loads.shipperdateactua
dbo.loads.usforwarderdatee
dbo.loads.billingcarrierlo
dbo.loads.haulingcarrierco
dbo.loads.haulingcarriertr
dbo.loads.interchangelocat
dbo.loads.interchangelocat
dbo.loads.consigneereferen
dbo.loads.statusavailablel
dbo.loads.consigneedatefin
dbo.loads.bookingcarrier_d
dbo.locations.Name AS customername, locations_1.Name AS shippername, locations_1.City AS shippercity, locations_1.State AS shipperstate, dbo.states.Country AS shippercountry,
dbo.states.country_mapcode
locations_2.State AS consigneestate, dbo.states_1.Country AS consigneecountry, dbo.states_1.country_mapco
locations_3.Name AS haulingcarriername, dbo.locations4.Name AS billtoname, dbo.bookingcodes.BookingCo
FROM dbo.loads LEFT OUTER JOIN
dbo.locations ON dbo.loads.CustomerID = dbo.locations.ID LEFT OUTER JOIN
dbo.locations4 ON dbo.loads.billtoid = dbo.locations4.ID LEFT OUTER JOIN
dbo.locations3 AS locations_3 ON dbo.loads.haulingcarrierid
dbo.loads_totalcharges ON dbo.loads.ID = dbo.loads_totalcharges.Loa
dbo.states_1 RIGHT OUTER JOIN
dbo.locations2 AS locations_2 ON dbo.states_1.Code = locations_2.State ON dbo.loads.consigneeid = locations_2.ID LEFT OUTER JOIN
dbo.states RIGHT OUTER JOIN
dbo.locations1 AS locations_1 ON dbo.states.Code = locations_1.State ON dbo.loads.shipperid = locations_1.ID LEFT OUTER JOIN
dbo.bookingcodes ON dbo.loads.BookingCarrierBo
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Are you running the query as Pass-Thru?
This query doesn't do anything special, other than return a lot of fields. Do you really need all of those fields in the result set?
The query looks simple enough. If your getting poor performance, my guess is its because you have not done proper indexing.
Its not perfect, but a good starting point is to index any column used in a foreign key. Other then that, fire up your query in SSMS, and hit Ctrl-L and the query plan will be shown along with any suggestions on it thinks might improve the query.
Its not perfect, but a good starting point is to index any column used in a foreign key. Other then that, fire up your query in SSMS, and hit Ctrl-L and the query plan will be shown along with any suggestions on it thinks might improve the query.
SQL Server and Azure optimization level depends on existing indexes at first. Then the optimizer uses statistics to decide about the query plan.
So if you provide list of indexes on your tables and also the query plan from your query then we may provide some optimizations hints.
Basically if all columns used in joins are indexed then we don't have many more options for optimization...
So if you provide list of indexes on your tables and also the query plan from your query then we may provide some optimizations hints.
Basically if all columns used in joins are indexed then we don't have many more options for optimization...
ASKER
No linked tables. The suggestions that the query is not all that complex and the suggestion to run the query directly in SSMS (much faster than the webpage output) led me to look at other sources of the slowness. Since this is a port from dedicated server to Azure I was only looking at SQL being the culprit. SSMS was returning the result set within 3-5 seconds. However, the webpage (Classic ASP) was taking 35+ seconds. Looking more closely at the webpage, it had additional code that took the SQL result recordset and looped through it executing further SQL for EACH record of the recordset.
Why? Because each "load" record has an unspecified number of associated records in a "loads_commissions" table for all the sales people participating in that "load". These commission records were looped through to create a delimited string value to represent all persons to be paid for that load. This string value is used in an unrelated accounting software.
Why? Because each "load" record has an unspecified number of associated records in a "loads_commissions" table for all the sales people participating in that "load". These commission records were looped through to create a delimited string value to represent all persons to be paid for that load. This string value is used in an unrelated accounting software.
Since the culprit is string concatenation for unknown records you can use the Power of SQL to do the dirty work
There many ways to accomplish that : https://sqlandme.com/2011/04/27/tsql-concatenate-rows-using-for-xml-path/
And also Concat_ws ... unfortunately I never had a case that needed such kind of output..but with some googling or question here you will get a solution that will get you lightning fast