I'm going to pose an interesting issue I have been tracking the past week or so.
The environment is Windows Server 2008 R2 Standard. It has Terminal Services installed and is used as a Terminal Server. The server is TS1. TS1 is a virtual guest running on a Windows Server 2008 R2 SP1 Enterprise host server. TS1 has 20gb of RAM allocated to it, 4 virtual cores, and has its .vhd stored on our SANS.
The server is used by roughly 50 users on a daily basis. Common applications are Office, programs that have a web interface and are hosted on the internet by a vendor, and various locally installed proprietary programs.
The problem that is being seen is high CPU usage. Getting into the details of it, when the CPU utilization is high, plateaued at 100%, users do experience a degraded experience. Office applications will hang up, keystrokes to the proprietary applications are delayed a second or two, and overall slowness in manipulating their session.
The high utilization is fairly consistent, with CPU usage dipping as low as 65%, generally staying above 90%. At certain points in the day, such as during early morning when users are connecting in, CPU usage stays peaked at 100%. Examining the usage these times show a consistent pattern. Many users have at least one process of iexplore.exe running at 6-15% cpu usage, never dipping below 5%. With a total of 50ish users this adds up to a maxed out CPU.
Examining several individual users sessions, their instances of internet explorer aren't appearing to be doing anything special. Browsing to common websites, www.msn.com
, etc. Some users were not even actively browsing, merely had the site up on their session.
Examining several processes of iexplore.exe that were continually pegging the CPU, they all end up looking like the same thing. I fired up the wonderful Process Explorer from Systernals, and did a little poking around on the processes and found that one thread is consistently the culprit, iertutil.dll!ordinal503+0x
e2. Even when the users are not using the browser but just have it up, this thread will go from a wrwaitforuser to ready back to wrwaitfuruser state over and over.
A few looks around on google didn't reveal much about iertutil.dll!ordinal503+0x
e2. Most results were posts about "Missing Ordinal###" when trying to launch IE. I did come across a software analysts post saying that in IE8.0, ordinal 503 is IsoRegisterWinProcComponen
Just to make sure it wasn't too high of user load, another server was setup, TS2, with the same OS, same applications, and about 1/3 of the users were migrated to use that server. On the new TS2 the issue is not apparent. The same users who were previously pegging out 5-15% of the CPU on their own were now not a problem.
Any ideas on what might be the source of the crazy high cpu utilization on this TS1?