bankonit
asked on
Windows Server 2008 R2 CPU Performance Issue
I'm going to pose an interesting issue I have been tracking the past week or so.
The environment is Windows Server 2008 R2 Standard. It has Terminal Services installed and is used as a Terminal Server. The server is TS1. TS1 is a virtual guest running on a Windows Server 2008 R2 SP1 Enterprise host server. TS1 has 20gb of RAM allocated to it, 4 virtual cores, and has its .vhd stored on our SANS.
The server is used by roughly 50 users on a daily basis. Common applications are Office, programs that have a web interface and are hosted on the internet by a vendor, and various locally installed proprietary programs.
The problem that is being seen is high CPU usage. Getting into the details of it, when the CPU utilization is high, plateaued at 100%, users do experience a degraded experience. Office applications will hang up, keystrokes to the proprietary applications are delayed a second or two, and overall slowness in manipulating their session.
The high utilization is fairly consistent, with CPU usage dipping as low as 65%, generally staying above 90%. At certain points in the day, such as during early morning when users are connecting in, CPU usage stays peaked at 100%. Examining the usage these times show a consistent pattern. Many users have at least one process of iexplore.exe running at 6-15% cpu usage, never dipping below 5%. With a total of 50ish users this adds up to a maxed out CPU.
Examining several individual users sessions, their instances of internet explorer aren't appearing to be doing anything special. Browsing to common websites, www.msn.com, www.yahoo.com, www.google.com, etc. Some users were not even actively browsing, merely had the site up on their session.
Examining several processes of iexplore.exe that were continually pegging the CPU, they all end up looking like the same thing. I fired up the wonderful Process Explorer from Systernals, and did a little poking around on the processes and found that one thread is consistently the culprit, iertutil.dll!ordinal503+0x
A few looks around on google didn't reveal much about iertutil.dll!ordinal503+0x
Just to make sure it wasn't too high of user load, another server was setup, TS2, with the same OS, same applications, and about 1/3 of the users were migrated to use that server. On the new TS2 the issue is not apparent. The same users who were previously pegging out 5-15% of the CPU on their own were now not a problem.
Any ideas on what might be the source of the crazy high cpu utilization on this TS1?
The environment is Windows Server 2008 R2 Standard. It has Terminal Services installed and is used as a Terminal Server. The server is TS1. TS1 is a virtual guest running on a Windows Server 2008 R2 SP1 Enterprise host server. TS1 has 20gb of RAM allocated to it, 4 virtual cores, and has its .vhd stored on our SANS.
The server is used by roughly 50 users on a daily basis. Common applications are Office, programs that have a web interface and are hosted on the internet by a vendor, and various locally installed proprietary programs.
The problem that is being seen is high CPU usage. Getting into the details of it, when the CPU utilization is high, plateaued at 100%, users do experience a degraded experience. Office applications will hang up, keystrokes to the proprietary applications are delayed a second or two, and overall slowness in manipulating their session.
The high utilization is fairly consistent, with CPU usage dipping as low as 65%, generally staying above 90%. At certain points in the day, such as during early morning when users are connecting in, CPU usage stays peaked at 100%. Examining the usage these times show a consistent pattern. Many users have at least one process of iexplore.exe running at 6-15% cpu usage, never dipping below 5%. With a total of 50ish users this adds up to a maxed out CPU.
Examining several individual users sessions, their instances of internet explorer aren't appearing to be doing anything special. Browsing to common websites, www.msn.com, www.yahoo.com, www.google.com, etc. Some users were not even actively browsing, merely had the site up on their session.
Examining several processes of iexplore.exe that were continually pegging the CPU, they all end up looking like the same thing. I fired up the wonderful Process Explorer from Systernals, and did a little poking around on the processes and found that one thread is consistently the culprit, iertutil.dll!ordinal503+0x
A few looks around on google didn't reveal much about iertutil.dll!ordinal503+0x
Just to make sure it wasn't too high of user load, another server was setup, TS2, with the same OS, same applications, and about 1/3 of the users were migrated to use that server. On the new TS2 the issue is not apparent. The same users who were previously pegging out 5-15% of the CPU on their own were now not a problem.
Any ideas on what might be the source of the crazy high cpu utilization on this TS1?
make sure your disconnect and idle session will be log off a particular time because idle & disconnected sessions may lead high CPU utilization and it use the system process (PID 4).
ASKER
@sharifmcd - Not sure what you mean specifically by "working directory". The users do have redirected folders for their desktop, documents, music, pictures, etc to another server on the network.
@sushil84 - The idle and disconnected sessions are set to be logged off after 1 hour of inactivity.
@sushil84 - The idle and disconnected sessions are set to be logged off after 1 hour of inactivity.
- If you login to the console (not rdp). Do you still face the same high cpu and slowness or it is only the rdp sessions experiencing this
- How does the system recover from it. Do you need to reboot it or it recovers on its own
- If you are saying that IE from each session is consuming around 5% of cpu then what is the behaviour if you kill these IE processes from the console session for the users does the system comes back to normal and runs fine
- How does the system recover from it. Do you need to reboot it or it recovers on its own
- If you are saying that IE from each session is consuming around 5% of cpu then what is the behaviour if you kill these IE processes from the console session for the users does the system comes back to normal and runs fine
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
The issue wasn't exactly resolved but mitigated.
check the "working directory" of the users
if they are using office application for instance on the terminal server TS1
i believe you should assign them a local folder in their PC's as a"working forlder" other wise as soon as the users start works and open files the RAM consumption will increase and since there a certain limit the RAM can be utilised then the system will start kick some "non-active" application to hard disk and this will utilize your CPU
I hope this answer your question