Solved

2012 R2 Hyper-V Slow HDD Performance

Posted on 2013-12-11
8
7,934 Views
Last Modified: 2016-11-23
Physical server - Dell PowerEdge 520 with H710 Raid Controller
Host OS - 2012 R2 fully updated and only Hyper-V service running
Host OS can copy data between logical volumes at high rates of speed - 200 MB/s upwards
Server has all SAS 10k RPM drives
Server is brand new
Guest OS 1 - 2012 R2 fully updated with AD, DHCP and DNS running
Guest OS 2 - 2012 R2 fully updated with Exchange 2013 running
Everything functions perfectly with regard to applications and core functionality.
Problem - data transfer to Guest OS (either including random test VM) is beyond terribly slow. Transfer speed is in the bytes and might sometimes get as high as 3-4 Mb/s.
Guest to Guest same thing. Host to Guest same thing.
Host B volume to Host C or D volume extremely fast.
VM Hard Drive is VHDX (tried fixed and dynamic)
Tried Generation 1 with IDE controller
Currently running Generation 2 with SCSI controller
Tried changing the channels for the SCSI drives and making sure they're unique
Currently updating Firmware and Drivers for RAID controller.
What am I missing??? I have never seen performance this poor. My test lab is the same setup and is lightning fast between Host and Guest.
0
Comment
Question by:CUBLA1
  • 5
  • 2
8 Comments
 
LVL 6

Expert Comment

by:RaithZ
ID: 39712965
What are the disk configurations?  Like a Raid 5 and then how is that divided?  Which volume are the guest machines stored on?  are the files being transferred from that volume, to the guest servers?  If so then you are reading and writing from the same logical partition/array which will significantly decrease disk performance.  It sounds like your host C and D drives are on separate arrays, which could explain why the transfer is so fast.
0
 

Author Comment

by:CUBLA1
ID: 39712987
Now I'm getting concerned that I have royally screwed myself because I've just spent 2 days prepping this entire network and just now realized the performance problem! What's weird is the entire time I was installing and configuring everything (even remotely) I was surprised at how smoking fast the server actually is. It just seems to be when moving data around that you notice something is wrong. Application performance is good. I just can't copy data from the old servers to the new ones because it said it would take 3945 hours!

I had to install Dell OMSA to review the RAID configuration. The server has (and my first post was wrong) six (6) SAS 300 Gb 15k RPM drives and 1 SATA 2 TB 7200 RPM.

Virtual Drive 0 - Raid 1 - 300 Gb Total (2 300Gb Drives) OS / Hyper-V
Virtual Drive 1 - Raid 5 - 900 Gb Total (4 300Gb Drives) VM and VHD storage for Hyper-V
Virtual Drive 2 - Raid 0 - 2 TB SATA Backup Drive (also SAS)

I am storing the Hyper-V VM and VHD locations on the same volume (VD-1 Raid 5) if that matters. The VM's run super fast. Everything runs super fast. I am using the 2012 R2 ISO at 4 Gb as the test file. If I copy from VD 0 to 1 no problem. Even if I copy it from VD 2 (the sata drive) to VD 0 or 1 no problem. If I login to any VM and copy it from anywhere to the VM it's slow as Christmas.
0
 

Author Comment

by:CUBLA1
ID: 39712995
Office File Transfer to VM Desktop
0
 

Author Comment

by:CUBLA1
ID: 39712998
This is the best its been yet. I updated RAID driver and firmware along with Bios and Intel Chipset. It seems to have improved dramatically compared to what it was before. It still seems to be bouncing from 20 mb/s to nothing up and down. Driver issue?
Win2012 File Transfer to VM desktop
0
Integrate social media with email signatures

Is your company active on social media? Do you also use email signatures? Including social media icons in your email signature is a great way to get fans for free. Let all your email users know you’re on social media quickly and easily, in a single click.

 
LVL 6

Expert Comment

by:RaithZ
ID: 39714520
The transfer that is slow, is the one that is going from one VM on the Raid 5 to any place else on the Raid 5.. or from any place to a VM... or from VM to VM right?  


All the tests you say are fast are going from one array to another, therefore the controller can read and write at the same time.

What you are seeing is not really abnormal for the situations described, though you might benefit from a RAID controller with more cache memory or a different controller.  

There are several factors that I think are contributing to your issue, each mentioned above.

So it sounds like its reading data, then writing it to the destination back and forth.  If you monitor the disk transfer at the source does it also go back and forth?
0
 

Accepted Solution

by:
CUBLA1 earned 0 total points
ID: 39772634
Actually it was very abnormal and Dell created a knowledge base article with regard to this particular problem. Seems there is a problem in 2012 R2 with Hyper-V and Virtual Machine Queues. We disabled VMQ on the physical NIC's and everything shot through the roof.
0
 

Author Closing Comment

by:CUBLA1
ID: 39781842
Forced to call Dell and have Tier 3 Microsoft support determine cause of problem. Took them 30 minutes to fix something I researched for 10 hours.
0
 
LVL 38

Expert Comment

by:Philip Elder
ID: 40469110
By default: No matter the server manufacturer Broadcom Gigabit physical NIC ports _MUST_ have VMQ disabled.

Broadcom for whatever reason chose to ignore the VMQ specification, since it was designed for 10GbE and higher, and enable it on 1GbE ports. This breaks everything as there is not enough network traffic on 1Gb for things to work. :(
0

Featured Post

Do email signature updates give you a headache?

Constantly trying to correctly format email signatures? Spending all of your time at every user’s desk to make updates? Want high-quality HTML signatures on all devices, including on mobiles and Macs? Then, let Exclaimer solve all your email signature problems today!

Join & Write a Comment

Every once-in-a-while, when you try to add a XenServer host to the System Center Virtual Machine Manager console, it will generate a certificate error, and the XenServer host will not be added to Virtual Machine Manager: If you are experiencing t…
First I will try to share a design of a Veeam Backup Infrastructure without Direct NFS Access backup. Note: Direct NFS Access backup transport mechanism is only available in Veeam v9 In above I try to design the Veeam Backup flow between i…
This video discusses moving either the default database or any database to a new volume.
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now