Terminal Server Performance Issues
Experts,
A few months back I installed a 2003 terminal server. The users keep complaining about how slow it is. Opening applications, switching back and forth between applications, typing faster then the server, etc. I have followed a few Microsoft KB articles but nothing seems to have helped. I originally thought this was related to an Outlook plugin for a web based CMS they use but now I'm not so sure. In the following KB article http://support.microsoft.com/kb/324446 the first two bullet points are spot on. Users are experiencing a lot of lag and slow/delayed response.
I made all the registry changes mentioned and rebooted the server. I attempted to turn on the "Enable Advanced Performance" option but my driver won't allow it.
Commonly run applications are Office 2007, Adobe Standard, Quickbooks Enterprise. All applications are licensed for Terminal Services. I have Vipre from Sunbelt Software installed and have confirmed is supported for TS. I have also followed Sunbelt's best practices documentation and given the TS a separate policy. I have removed all scans and tried to make it as lean as possible.
Any help would be greatly appreciated. Please let me know any other information you may need. I can provide screenshots, etc. upon request.
Server specs:
Intel Xeon E5520 @ 2.27GHz
16GB of RAM
I have attached an image from the Task Manager. This is what it looks like all day. When user's call in there does not seem to be a rises in processor or memory utilization. There are approx. 30 users setup but typically not all logged in at the same time.
Task-Manager-Base.png
A few months back I installed a 2003 terminal server. The users keep complaining about how slow it is. Opening applications, switching back and forth between applications, typing faster then the server, etc. I have followed a few Microsoft KB articles but nothing seems to have helped. I originally thought this was related to an Outlook plugin for a web based CMS they use but now I'm not so sure. In the following KB article http://support.microsoft.com/kb/324446 the first two bullet points are spot on. Users are experiencing a lot of lag and slow/delayed response.
When a user types in a document, characters do not appear on the screen until this pause has ended
.
Keyboard input and mouse input are queued, but they are not processed until this pause has ended.
I made all the registry changes mentioned and rebooted the server. I attempted to turn on the "Enable Advanced Performance" option but my driver won't allow it.
Commonly run applications are Office 2007, Adobe Standard, Quickbooks Enterprise. All applications are licensed for Terminal Services. I have Vipre from Sunbelt Software installed and have confirmed is supported for TS. I have also followed Sunbelt's best practices documentation and given the TS a separate policy. I have removed all scans and tried to make it as lean as possible.
Any help would be greatly appreciated. Please let me know any other information you may need. I can provide screenshots, etc. upon request.
Server specs:
Intel Xeon E5520 @ 2.27GHz
16GB of RAM
I have attached an image from the Task Manager. This is what it looks like all day. When user's call in there does not seem to be a rises in processor or memory utilization. There are approx. 30 users setup but typically not all logged in at the same time.
Task-Manager-Base.png
Lee W, MVP
What kind of disk subsystem do you have? Try using performance monitor (system Monitor) to verify how the disk subsystem is responding.
ASKER
Just created a log for the disks. Hopefully that will give me some information on the disks for you. I currently have a PERC 6/i SCSI Disk Device listed under Disk Drives in Device Manager.
Does the performance lag increase as the users increase?
If only one user is logged in do they see the performance issues?
Are all your drivers up to date?
If only one user is logged in do they see the performance issues?
Are all your drivers up to date?
Are the users connecting from a remote site? Could it be a slow connection?
One way to tell if it's their connection: shadow one of the users while they're experiencing the problem. If you can see the lag as you are remote controlling their session through TSAdmin, then it's on the server. If response time looks good to you, but it still really slow for them: it might be a bandwidth issue.
When the connection is slow, the image of the remote session sent back down to the user's PC can be delayed whilch will cause it "repaint" slowly on their screen which will make it appear to have a significant lag to them. But, it could still look good to you.
If that's the case, I would test on their network or internet connection to see what kind of response time they are seeing. Talking with the ISP can also be helpful because they would be able to tell you if that location is maxing out their bandwidth. We've seen behavior like this when multiple users are streaming video and music or downloading large files (like movies). We've also seen this when one or more computers at the remote site had some sort of virus that was ate up all their bandwidth.
Good luck,
Alicia
One way to tell if it's their connection: shadow one of the users while they're experiencing the problem. If you can see the lag as you are remote controlling their session through TSAdmin, then it's on the server. If response time looks good to you, but it still really slow for them: it might be a bandwidth issue.
When the connection is slow, the image of the remote session sent back down to the user's PC can be delayed whilch will cause it "repaint" slowly on their screen which will make it appear to have a significant lag to them. But, it could still look good to you.
If that's the case, I would test on their network or internet connection to see what kind of response time they are seeing. Talking with the ISP can also be helpful because they would be able to tell you if that location is maxing out their bandwidth. We've seen behavior like this when multiple users are streaming video and music or downloading large files (like movies). We've also seen this when one or more computers at the remote site had some sort of virus that was ate up all their bandwidth.
Good luck,
Alicia
ASKER
There is a remote site over a point-to-point T1 but there are also many users on the same local network that are experiencing the sames problems. I don't see a slow connection being an issue here.
ASKER
I can't tell yet if users are experiencing these issues during off peak hours yet. I am asking users to pay more attention to that going forward. Below is a screenshot of the performance monitor if that helps anyone. I am more than happy to provide any other data needed.
PerfMon.png
PerfMon.png
Well it can be MANY things.
First of all is this a 32-bit or 64-bit OS? If this is 32-bit, the only way to see 16GB is by using PAE what can actually do way more harm then good.
Then, are you running CTFMON.EXE with Office? If you are, that is also a known problem. Typical symptoms are slowdowns like you are seeing.
My first try would be to disable PAE on boot.ini and run as a normal 32-bit system with 4GB and see what happens. And of course check for CTFMON.EXE.
Also check what is on the Run key to see what processes are launched automatically with each session. Some do nasty stuff.
And if you have any AntiVirus on the TS, try disabling it for testing purposes.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
First of all is this a 32-bit or 64-bit OS? If this is 32-bit, the only way to see 16GB is by using PAE what can actually do way more harm then good.
Then, are you running CTFMON.EXE with Office? If you are, that is also a known problem. Typical symptoms are slowdowns like you are seeing.
My first try would be to disable PAE on boot.ini and run as a normal 32-bit system with 4GB and see what happens. And of course check for CTFMON.EXE.
Also check what is on the Run key to see what processes are launched automatically with each session. Some do nasty stuff.
And if you have any AntiVirus on the TS, try disabling it for testing purposes.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
ASKER
This is a 32-bit OS with the PAE switch. I also see the CFTMON.EXE running. I am currenlty looking at removing this first. I am reading KB 823586 for this purpose but still looking. Thanks for this comment.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
You think the PAE switch is more likely the problem then CTFMON.EXE? It's currently running on everyone's profile. I can end the process and make the proper changes according to http://support.microsoft.com/kb/823586. Your thoughts?
Thanks again.
Thanks again.
ASKER
The OS is an Enterprise edition but I'm not sure yet if that makes a difference with the PAE switch.
Start with the CTFMON as regardless of where the issue is, CTFMON is really not recommended to be running on any TS, 32-bit or not.
Then go for the PAE change as this as well is not really a good thing with TS as well.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
Then go for the PAE change as this as well is not really a good thing with TS as well.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
ASKER
Will do. Tonight I will remove CTFMON.EXE and let you know if the morning. I'll update you in a day or so.
ASKER
Did not have luck removing the CTFMON.EXE from starting. I am still looking into this issue. The KB article is not at all accurate when it comes to Office 2007. I was able to find some articles but they did not remove the service. I also attempted to unregister some DLLs related to CTFMON.EXE but this also failed.
Any ideas how to remove the CTFMON.EXE from Windows Server 2003 with Office 2007?
Any ideas how to remove the CTFMON.EXE from Windows Server 2003 with Office 2007?
Have you tried simply rebooting the box, not allowing anyone to logon, login as admin and rename CTFMON.EXE to something else?
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
ASKER
Renamed CTFMON.EXE last night in C:\Windows\System32 directory but it came back. I am still working on this issue. I have been able to make more changes and will be rebooting the server tonight.
I will see if getting rid of this service makes a difference. If not, I will then remove the /PAE switch in the boot.ini file.
Sorry this has taken so long.
I will see if getting rid of this service makes a difference. If not, I will then remove the /PAE switch in the boot.ini file.
Sorry this has taken so long.
ASKER
Through many attempts it seems impossible to remove the CTFMON.EXE from running on this server. I simply cannot stop this from running.
Read this thread please:
http://www.brianmadden.com/forums/t/21336.aspx
As you can see, applying NTFS permissions to DENY it for regular users should do the trick as well.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
http://www.brianmadden.com/forums/t/21336.aspx
As you can see, applying NTFS permissions to DENY it for regular users should do the trick as well.
Cláudio Rodrigues
Microsoft MVP - Remote Desktop Services
Citrix CTP
ASKER
This seems to have stopped CTFMON.EXE from running. I will monitor this. Next step will be to remove the /PAE switch from boot.ini.
Thanks for your help.
Thanks for your help.
ASKER
Yes it has been a while but I am working on this server in questions still.
I have removed the /PAE switch from the boot.ini but still am seeing 16GB of RAM and Physical Address Extension under the properties of my computer.
Any thoughts?
Here is what my boot.ini consists of:
[boot loader]
timeout=30
default=multi(0)disk(0)rdi
[operating systems]
multi(0)disk(0)rdisk(0)par
I have removed the /PAE switch from the boot.ini but still am seeing 16GB of RAM and Physical Address Extension under the properties of my computer.
Any thoughts?
Here is what my boot.ini consists of:
[boot loader]
timeout=30
default=multi(0)disk(0)rdi
[operating systems]
multi(0)disk(0)rdisk(0)par
ASKER
I made the changes provided and did notice some improvement but after more research the real problem seems to be the Outlook plugin for the CRM solution. That software company is having quite a few problems with this plugin.
Thanks for all the help.
Thanks for all the help.