Solved

Slow Response Times after move to New Server IIS6 to IIS7.5

Posted on 2013-06-23
22
2,341 Views
Last Modified: 2014-06-12
Hi,

I recently moved to a new server with Windows 2008, IIS7.5 and SQL Server 2008, the new server also has 24 CPUs against the previous 8 and 32GB Ram against 24. Since the move the site has seen response times drop from 3 to 5 seconds on average and in some cases on a page reading info from a stored procudre just to display it from 3 to 8 seconds.

I have run a response time checker it shows an increased WAIT time in a waterfall chart from 1 to 5.1 seconds.

I have tried as much as I can find, including changing pipelines, 32/64 bits app pool etc.

Any ideas welcome?

Cheers
Karl
0
Comment
Question by:Kbba04527
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 14
  • 8
22 Comments
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39269732
Do you have experience with sql profiler traces? I would recommend running a profile trace to see what the sql server response times are (to see if the delay is on iis or sql server's side).  Has anything else changed ? (ie. physical to vm, more roles on the server, etc)
0
 

Author Comment

by:Kbba04527
ID: 39269757
Hi,

I don't have experience with SQL profiler, but this seems the perfect time to learn. I'll have a read up on that tomorrow.

I looked at the roles and the old server had the added roles of DNS Server and Application Server, do you think this could be an issue and if I added them could it be problematic to a live server?

Cheers
Karl
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39269812
I personally wouldn't put DNS on a web server because of the increase in the attack surface area. Especially with DoS if it's authoritative for your domain. In terms of performance, I don't think it would be a huge issue. My concern is mostly with sql, as it's a bit of a memory hog and could starve your other roles.  I would also check how much memory sql server is currently consuming. If it's a lot, you may want to consider capping it (capping it probably isn't a bad safeguard regardless).

Capping memory: http://msdn.microsoft.com/en-us/library/ms178067.aspx

SQL server profiler: http://msdn.microsoft.com/en-us/library/ms181091.aspx

As an aside, I should have previously mentioned that profiler traces can have severe impacts on sql performance. The more you capture, the more of an impact. And try to be as restrictive as possible with your filters.
0
NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

 

Author Comment

by:Kbba04527
ID: 39270546
Hi,

I profiled both servers for the same query and the profiler response times to run the query were

NEW : 00:00:00.0565171
OLD : 00:00:00.0289281

Would this be enough to cause 3 seconds to go to 7/8 seconds?

Cheers
Karl
0
 

Author Comment

by:Kbba04527
ID: 39270920
I think I have made a bit of a breakthrough.

On the page that's running slow I increased it to include more records, so that response time was hitting 10 seconds then took out the code on the page that writes to the screen, this reduced the load time to half a second. It still loops the records, but doesn't write to screen.

Any ideas on why it would be so slow?

Cheers
Karl
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39270958
What is the current resource use looking like on ram/CPU side, out of curiosity? I imagine you have plenty of wiggle room, but just want to make sure.
0
 

Author Comment

by:Kbba04527
ID: 39270968
The CPUs aren't really moving off 1%, max 15%. I'm using 5.5GB of 32GB Ram.

I have just installed Dynamic Compression and no change.
0
 

Author Comment

by:Kbba04527
ID: 39271991
Hi,

I installed DataWizard SQL Profiler and it's showing me highest queries by duration to run and this query sits at number 1 with an average run time of 4.2 seconds, so it is SQL that's causing the delay

I'm a bit stuck now, whats the best way to work out why?

Cheers
Karl
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39272038
Try looking at the execution plan of the SQL query when you execute it.  Is it an ad hoc query or a stored procedure?
0
 

Author Comment

by:Kbba04527
ID: 39272091
Stored Procedure. I'll paste in info shortly
0
 

Author Comment

by:Kbba04527
ID: 39272100
"select ID,orderDate,MTYOrders,MTYPostage,FFOrders,FFPostage,FMOrders,FMPostage, MTYOrders+MTYPostage+FFOrders+FFPostage+FMOrders+FMPostage AS TOTAL,[UPDATE],[Note] from viewReporting
where orderdate >= @date_from
and orderdate <= @date_to
order by ID desc"		

Open in new window

0
 

Author Comment

by:Kbba04527
ID: 39272167
On a side note, my site response time as a whole has dropped between 1-2 seconds as all pages feature some sql.
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39272306
Is your old server still up and running, by chance?  I was hoping that you might be able to take a look at the execution plans in both environments to see if there is anything different in the new vs. old. And how did you migrate the database to the new server?
0
 

Author Comment

by:Kbba04527
ID: 39272409
The old server is still up and running. To migrate I did a backup and restore. I'll look over the execution plan now.
0
 

Author Comment

by:Kbba04527
ID: 39272542
Old Server
Old Server
New Server
New Server
The old one is the faster one that's moaning.
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39272597
That's really confounding...  Especially with the missing index on the old server...  You may want to support the query with an index (i'd start with a nonclustered index on orderdate and then you can look at including some of the other columns - unfortunately, indexing is a bit of an art and I am not as practiced as I'd like to be before I start definitively spouting out indexes) to try and avoid the table scan, however the execution plan is showing that you're comparing apples to apples, so...  The only thing I can think of is either: you're bottlenecked on the hardware in some way or there is a lot more data in the new server's viewReport table.  A table scan will touch every row of a table regardless of whether or not it qualifies.
0
 

Author Comment

by:Kbba04527
ID: 39272627
I have added an Index to the new one on ID and Order Date to no avail, the funny thing is if you run the query against the stored procedure and say for example 01/01/2012 to 01/05/2013 it takes about 10 seconds. If you go into the code and remove just the part that writes to the screen and leave the loop running through it, it runs in about .5 seconds.

The codes seems slower site wide, but only hugely on pages which write a lot to the screen.
0
 
LVL 8

Expert Comment

by:didnthaveaname
ID: 39281573
Ugh.  I was honestly hoping it was going to be a SQL issue, but I think it may be something with IIS, which I am freely going to admit I am not as strong on.  I'll poke around in hopes of finding something, but hopefully someone else may have some new ideas.  Is this new server a virtual server or no?
0
 

Author Comment

by:Kbba04527
ID: 39281598
Hi,

Cheers for all your help.

This is not a Virtual Server.

Cheers
Karl
0
 
LVL 8

Accepted Solution

by:
didnthaveaname earned 500 total points
ID: 39281712
You may want to look into this: http://www.iislogs.com/steveschofield/troubleshooting-iis-7-network-performance-issues-and-tcp-chimney-offload-receive-side-scaling-and-network-direct-memory-access.  I've seen a lot of articles talking about this feature causing a lot of issues, and the default behavior changed to be enabled in 2008 and it would have an impact network traffic too.

Edit: Also, and I hate using a competitor forum to answer an answer here, but it is another thing to look at (as he seemed to be having the same obscure issue you are with the exact same circumstances): http://stackoverflow.com/questions/12474880/iis-7-5-on-windows-server-2008-r2-slow (he mentions in a followup that it was ISAPI filters)
0
 

Author Comment

by:Kbba04527
ID: 39281767
Hi,

That last link, I tested the code in asp.net vb and without sessions it was 2 seconds, with it 12 seconds - hopefully a breakthrough.

I now need to look at ISAPI Filters. Will keep you posted.

Cheers
Karl
0
 

Author Comment

by:Kbba04527
ID: 39281843
Hi,

I removed all the ISAPI filters to no avail and then found this - http://serverfault.com/questions/249262/why-do-web-sites-in-iis7-7-5-have-asp-net-isapi-filters-enabled-by-default - so I popped them back on.

But it gives me a good starting point to look at the speed issue with the session test.

Cheers
Karl
0

Featured Post

Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A phishing scam that claims a recipient’s credit card details have been “suspended” is the latest trend in spoof emails.
Resolving an irritating Remote Desktop connection that stops your saved credentials from being used.
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will walk an individual through the process of transferring the five major, necessary Active Directory Roles, commonly referred to as the FSMO roles to another domain controller. Log onto the new domain controller with a user account t…
Suggested Courses

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question