Have a hyper-v environment where all physical hosts connect to a Powervault 3800f with 10x15k SAS in RAID10 via fibre channel - one virtual disk per host, 4 hosts.
On the SAN performance logs I'm seeing latency spikes of around 50ms on each virtual disk, but average latency is around 2ms.
On the VMs at the same time I'm seeing latency spikes of 600ms - once I even saw 1100ms.
This tells me that something is going wrong between the SAN and the VM.
Write cache hit is constant at 100%, read cache hit is usually around the 70% mark - read % is usually below 40%.
Maximum combined IOPS of two raid controllers is around 9k.
On the VMs memory and processor usage is unremarkable - high paging on all VMs though (using dynamic memory - maybe stick to fixed?)
The VM with a particular issue has processor usage of around 50% (25 processors) - so split over NUMA nodes.
Where should I be looking for the delay between the SAN and the VM ?