Disk I/O on virtual SBS 2008
I have several SBS installations as virtual machines on VMware ESX and ESXi ranging from 3.5 - 4.1.
One of the installations is using a high amount of I/O according to the vCenter Performance monitor. How can I tell which process is using the disk so much?
This installation have 7GB of RAM. Please before you tell me that is not enough It is by far not the lowest of all the installations. I have one installation (both are SBS 2008) which had only 5GB and is using 1/20th the I/O.
Please understand I also think this should have more RAM and I have gotten permission to order the RAM but what I do not want to happen is to install the RAM and still have the same issue.
One of the installations is using a high amount of I/O according to the vCenter Performance monitor. How can I tell which process is using the disk so much?
This installation have 7GB of RAM. Please before you tell me that is not enough It is by far not the lowest of all the installations. I have one installation (both are SBS 2008) which had only 5GB and is using 1/20th the I/O.
Please understand I also think this should have more RAM and I have gotten permission to order the RAM but what I do not want to happen is to install the RAM and still have the same issue.
what disk subsystem is the underlying datastore?
ASKER
hanccocka
HP P400 array controller with 512MB BWBC. 2 Array's 1 SAS and 1 SATA on the same controller.
This is the only server on the Host that is having an I/O issue. It is on the SATA array. 10 VM on this host most of them very low usage.
MPECSInc
SBS 2008 and Exchange 2007 SP3 are I/O hungry
I do know this but as a comparison 1 Install on ESX 3.5 5GB of RAM is running average 566 KBps the SBS with the issue is running 26,542 KBps.
There is something wrong here. I will look at the links you have provided. Right now I have to go take a test on HP StorageWorks cert. Aaaaaaaaa :-)
HP P400 array controller with 512MB BWBC. 2 Array's 1 SAS and 1 SATA on the same controller.
This is the only server on the Host that is having an I/O issue. It is on the SATA array. 10 VM on this host most of them very low usage.
MPECSInc
SBS 2008 and Exchange 2007 SP3 are I/O hungry
I do know this but as a comparison 1 Install on ESX 3.5 5GB of RAM is running average 566 KBps the SBS with the issue is running 26,542 KBps.
There is something wrong here. I will look at the links you have provided. Right now I have to go take a test on HP StorageWorks cert. Aaaaaaaaa :-)
Is your controller set to 75% write, 25% read.
Do you have RAID 10 configured?
SATA disks are very poor for IOPS, SAS are better.
How many disks in the RAID sets?
more disks = more spindles = more performance?
Do you have RAID 10 configured?
SATA disks are very poor for IOPS, SAS are better.
How many disks in the RAID sets?
more disks = more spindles = more performance?
There are a lot of variables involved when it comes to I/O and SBS. Different firms will utilize resources in very different ways.
We would not run SATA anywhere near a virtualization setup.
Check your Disk Queue Lengths on both host and in the VM. Note which processes are running using the tools mentioned. You will be able to figure out very quickly where your bottleneck is.
http://blog.mpecsinc.ca/2011/10/hyper-v-performance-appropriate-disk.html
Philip
We would not run SATA anywhere near a virtualization setup.
Check your Disk Queue Lengths on both host and in the VM. Note which processes are running using the tools mentioned. You will be able to figure out very quickly where your bottleneck is.
http://blog.mpecsinc.ca/2011/10/hyper-v-performance-appropriate-disk.html
Philip
ASKER
hanccocka
Is your controller set to 75% write, 25% read.
How can you tell? I have never see an option for this.
I know the differences between SATA and SAS and how all that works. I am relativity sure this is not a HDW issue. I know it would be best to have all SAS and RAID 10 would be great but there are many time especially with the SMB customer you do not always have the luxury of implementing the optimum system.
I feel confident this system has more power than they need I think we have a rouge software issue. Will implement these motioning tools I sure we can track it down.
Thanks for all your help I will let you know.
Is your controller set to 75% write, 25% read.
How can you tell? I have never see an option for this.
I know the differences between SATA and SAS and how all that works. I am relativity sure this is not a HDW issue. I know it would be best to have all SAS and RAID 10 would be great but there are many time especially with the SMB customer you do not always have the luxury of implementing the optimum system.
I feel confident this system has more power than they need I think we have a rouge software issue. Will implement these motioning tools I sure we can track it down.
Thanks for all your help I will let you know.
You would need to boot the server from the Smart Array bootable CDROM, run the Array Configuration Utility, select the Array Configuration and alter the BBWC configuration.
Disable AV if present.
Disable AV if present.
ASKER
hanccocka Good information Thanks!!
if you've never configured it......read this EE users comments....
https://www.experts-exchange.com/questions/27269019/Why-slow-disk-to-disk-copy-in-VMWare-ESXi-server-4-1.html
Oh My God
What a difference !
Both copies of 12 Gb now down to 3 mins (previously 18 and 25 mins).
And that's from NFS to local datastore
Local to Local - just 2.5 minutes. Amazing
Thanks Hancoccka
https://www.experts-exchange.com/questions/27269019/Why-slow-disk-to-disk-copy-in-VMWare-ESXi-server-4-1.html
Oh My God
What a difference !
Both copies of 12 Gb now down to 3 mins (previously 18 and 25 mins).
And that's from NFS to local datastore
Local to Local - just 2.5 minutes. Amazing
Thanks Hancoccka
This must be an ESXi thing because we do not see these slow disk issues on our configurations with Hyper-V at all.
A typical RAID 10 array with 4 or 6 15K SAS spindles should do at least 200MB/Second or more. 300MB/Second is more in line with that configuration.
SATA 7200 in RAID 10 would be 160MB/Second on 4 spindles while 6 spindles may pull about 200MB/Second.
Philip
A typical RAID 10 array with 4 or 6 15K SAS spindles should do at least 200MB/Second or more. 300MB/Second is more in line with that configuration.
SATA 7200 in RAID 10 would be 160MB/Second on 4 spindles while 6 spindles may pull about 200MB/Second.
Philip
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Ah, IC.
We don't work with HP. All servers we deploy have RAID +BBU and are configured out of the box with performance in mind.
So, learned something. :)
Philip
We don't work with HP. All servers we deploy have RAID +BBU and are configured out of the box with performance in mind.
So, learned something. :)
Philip
ASKER
OK MPECSInc I downloaded this tools. How do I use them to tell which program is using up all the Disk Activity?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks for all your help guys I think I found the issue. I believe it is the SQL Server Monitoring Incident.
Ran this query to add the Index and that seem to have taken care of it. I will monitor for the weekend to be sure.
http://social.technet.microsoft.com/Forums/en/smallbusinessserver/thread/d195baac-da8b-4387-9079-c55d5e1879b4
Ran this query to add the Index and that seem to have taken care of it. I will monitor for the weekend to be sure.
http://social.technet.microsoft.com/Forums/en/smallbusinessserver/thread/d195baac-da8b-4387-9079-c55d5e1879b4
ASKER
The SBS went from 19,000 KBps last Saturday 5AM - 5PM to 800 KBps this Saturday 5AM - 5PM. What a difference.
What are your VM Disk Queues and host Disk Queues like?
ProcessExplorer and ProcessMonitor are good places to start:
http://technet.microsoft.com/en-us/sysinternals/bb896645
http://technet.microsoft.com/en-us/sysinternals/bb896653
Philip