Server 2008 Standard 32-bit - Terminal Server - Programs install VERY slow
I have a Server 2008 32-bit Standard box running as a VM on VMWare ESX 4.0 This machine is set up with 2 CPU's and 4GB of RAM. About 3 months ago I noticed that everything I install on this machine takes FOREVER. For a standard Adobe Reader X update it will run for about 3 hours. It is horrible. I don't see any services consuming high amounts of memory. The processor seems to be sitting at a reasonable percentage when installing. I just can't figure it out. I'm doing Windows Updates now and for 10 standard Windows updates this will probably take 3+ hours. My other machines that are set up very similar will go through these updates in 30 minutes. Has anyone seen anything like this?
Aaron Tomosky
If it's not out of memory and having to swap to disk, sounds like an io problem. Local drive? Iscsi?
What specifications are the ESX Host Server, Memory, CPU and Underlying storage system?
Could be Processor, Memory or disk storage botteneck?
Could be Processor, Memory or disk storage botteneck?
What are the memory allocations for all vms on the server and what is the total real ram on the server? If each vm added up plus what esxi uses is more than the real ram...
ASKER
This is a VM running on iSCSI. I can't see it being disk related because there are 5 other VM's on this physical host and all run great.
The Host is HP Proliant G5 with dual quad core 2.833Ghz with 32GB RAM all hooked into an iSCSI HP dual node SAN system. I truly beleive this is something in the OS because I am doing all my server updates today and all the other servers are fine.
The Host is HP Proliant G5 with dual quad core 2.833Ghz with 32GB RAM all hooked into an iSCSI HP dual node SAN system. I truly beleive this is something in the OS because I am doing all my server updates today and all the other servers are fine.
ASKER
The Host itself has 32GB and right now the utilization is running at 17GB. I don't allow any server to get higher than 20GB utilization in my VM Cluster. The processor of the host is running 10% of its limit. I've moved this VM between hosts to see if it does anything for this issue and it does not. Everything else runs great on this box. This is a Terminal Server that hosts up to 30 sessions all day long without complaints.
My only issue is installing software or updates. Otherwise it runs great.
My only issue is installing software or updates. Otherwise it runs great.
Have you disabled Anti-Virus?
What are the performance statistics for Read and Write Latency, CPU, Memory?
Task Manager and Resource Monitor in the VM?
Veeam Monitor for free
If you want a quick tool, to help you with wanting to know all the individual elements, then I would suggest using Veeam Monitor Free Edition
Source
http://www.veeam.com/esxi-monitoring-free.html.
If you want to get down and dirty with the servers, then I would recommend the following documents
Performance Monitoring and Analysis
Guest-based performance monitoring is an inaccurate and unhelpful means of evaluating performance in virtual deployments. See Guest-based Performance Measurement for more inforamtion. Monitoring and analysis of VMware ESX Server should be performed with esxtop and VirtualCenter.
esxtop is the tried-and-true means of collecting every performance stat needed and making it available in a way that is conducive to analysis. The best source of information on launching esxtop can be found in the Resource Management Guide (page 159).
1. Check and correct CPU utilization: CPU Performance Analysis and Monitoring
2. Identify memory bottlenecks and remove: Memory Performance Analysis and Monitoring
3. Characterize storage performance and correct: Storage Performance Analysis and Monitoring
Within each of these articles are techniques for using counters from VirtualCenter and esxtop. Information on those counters is provided in
vCenter Performance Counters
esxtop Performance Counters
Also, note that, while useless in collecting performance data, Perfmon can help with analysis of large esxtop output files.
Using Perfmon for esxtop-based Performance Analysis
What are the performance statistics for Read and Write Latency, CPU, Memory?
Task Manager and Resource Monitor in the VM?
Veeam Monitor for free
If you want a quick tool, to help you with wanting to know all the individual elements, then I would suggest using Veeam Monitor Free Edition
Source
http://www.veeam.com/esxi-monitoring-free.html.
If you want to get down and dirty with the servers, then I would recommend the following documents
Performance Monitoring and Analysis
Guest-based performance monitoring is an inaccurate and unhelpful means of evaluating performance in virtual deployments. See Guest-based Performance Measurement for more inforamtion. Monitoring and analysis of VMware ESX Server should be performed with esxtop and VirtualCenter.
esxtop is the tried-and-true means of collecting every performance stat needed and making it available in a way that is conducive to analysis. The best source of information on launching esxtop can be found in the Resource Management Guide (page 159).
1. Check and correct CPU utilization: CPU Performance Analysis and Monitoring
2. Identify memory bottlenecks and remove: Memory Performance Analysis and Monitoring
3. Characterize storage performance and correct: Storage Performance Analysis and Monitoring
Within each of these articles are techniques for using counters from VirtualCenter and esxtop. Information on those counters is provided in
vCenter Performance Counters
esxtop Performance Counters
Also, note that, while useless in collecting performance data, Perfmon can help with analysis of large esxtop output files.
Using Perfmon for esxtop-based Performance Analysis
Are you using Jumbo Frames to support iSCSI?
Windows Updates can be slow, on Windows 2008 systems, but this is usually down to slow datastore.
Are all the other 5 VMS, Windows 2008?
Windows Updates can be slow, on Windows 2008 systems, but this is usually down to slow datastore.
Are all the other 5 VMS, Windows 2008?
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
ASKER
Yes I have jumbo frames enabled. Some servers are 2008 some are 2003. Its an equal mix I would say.
ASKER
The print driver thing would actually make sense. I did install the Universal print driver on this system and as I watched the updates get installed, I see Spoolsvc.exe come up a few times.
I also find it very odd, that right now I have a "virtual machine cpu usage warning" going on. It is a yellow warning now, but doing simple Windows updates do not cause this on any other server. I think its something in the OS that is causing installs to need to work very hard.
I'm going to look into this Universal print driver thing.
I also find it very odd, that right now I have a "virtual machine cpu usage warning" going on. It is a yellow warning now, but doing simple Windows updates do not cause this on any other server. I think its something in the OS that is causing installs to need to work very hard.
I'm going to look into this Universal print driver thing.
ASKER
So I opened up ProcessMonitor and watched what MSIExec is doing. It is exactly what that article about the print driver explain. There are thousands of entries of MSIExec going through HewlettPackard Registry entries. I have to do further research to see how this can be fixed!
Windows 2008 and WIndows 2008 R2 Updates always cause this on our servers and Clients servers, when using Windows Updates.
that's the CPU Warning lights to light.
ASKER
Ah good, its just odd because this is the only server that does this. The others even with less memory don't warn about anything. This server just seems to need to work harder to do everything. I think the print driver stuff may have something to do with it.