New Hyper-V VM's log on is very slow

Are they on the same VLAN?

I didn't u a vlan

Ok, then you have them in same IP range? Let's say 10.10.10.x or not? Are all servers on same operating system?

Yes and yes

Both of the slow VM’s have web server as one of their roles.  Could that be it?

are you connecting via server name or IP Address?

name

if you use IP Address, does it make anty difference?

I just tried IP, same speed

if you shutdown the other VMs, does the access speed up?

The web server is not the issue.  I have another location with a VM and it is a web server.  Logs on fast

Shutdown made no difference

Could the physical nic they are connected to have anything to do with it?

Not really. What response times you get if you ping those servers?

all three under 1 millisecond

Then is something on those 2 servers, are all in same OU? Do you have GPO's that are applied to those 2 servers and not to "fast" server?

Different  ou.  No GPO's

where is the delay, waiting for the RDP session, or the actual login?

if you access the console, is the delay the same?

different users cause the same delay?

roaming profiles? cached profiles?

Actual login.
If I open it from the Hyper-V mgr it is fast
I'll try another user
Not roaming.  Not sure what a cached profile is?  But it's the same user for all so that shouldn't be the issue.

As a test I just added a new VM and it has the same login speed issues.  

I'm not so much concerned about the login time, but I am concerned that if I add applications to run on the servers there will be a lag in the apps performance

Is there a role that is "not" on the slow servers that could cause this?

One thing that is different with the slow VM's is when I log on to them via RDP I get the Welcome screen.  When I log to the fast VM it goes straight to the desktop.

Still need advice on this.

In my experience, slow logons (along with slow desktops and overall sluggishness) in a VM are almost always disk I/O bound - that is, the disk subsystem (where temp files, ntuser.dat, page file, etc,etc) are dragging.

Once you get to the (yawn) desktop, open Task Manager- Resource Manager and look at the disk section.  If it's taking more than 99ms for any process to get any disk request, then you have a disk bottleneck.  (It's ok for background tasks to take longer to get their data, but generally I like fast disks < 40ms per request).

It's easy to overcommit disks in a HyperV, especially if the host OS is residing on the same spindles as the guests.

It's possible to overcommit the host CPU too, which will reveal itself by looking at task manager of the host.

Welcome machines are windows 2008?

and others are?

All the slow machines were built after the fast machine?

Maybe the fast machine has update integration components.

All 3 are 2008 R2.

The slow machines were built first, that's the puzzling part.  

The fast server is a DC

Please explain "update integration components"

After reviewing the task manager for the VM and the host.  The slowest disk response was 12ms for either.  The VM is using 17% of it memory and the host is using 56%.  Both had a CPU usage under 3%

Are you 'doing things' to stress the VM out?  Perhaps having one session with Resource Manager open and watching it while another RDP session logs on would expose the delays..

Currently the VM's aren't performing any tasks, so nothing would be stressing them out.  This is the first time I've check resource manager so that wouldn't be the cause.  

Don't you thinks it's odd the the last VM added is the fast one and also that it is a DC?

What are the DNS settings on the machines? DHCP or manually configured. I suspect the delayed machines maybe on DHCP or using ISP DNS server details as opposed to configured to look at the internal DNS.

Does that pinpoint your issue?

If not the NIC cards configured on your VMs, are they synthetic or legacy? Can you flip them all over to synthetic, any improvement?

They are static.  How do I tell if they are synthetic or legacy?  Are you referring to the the physical nic or virtual?

This should tell you the way

http://windowsitpro.com/t-sql/hyper-v-synthetic-devices

Physical NIC are neither legacy nor synthetic.

These two are Hyper-V terms

Can you tell me which of these two Virtual NIC settings are correct?
local-10.png
local-area-11.png

Allow me to re-state your original question to make sure we're not barking up the wrong tree..  "1 of them opens very quickly via RDP, the other two take approximately 2 minutes"

Is this an RDP delay, or is the delay in actually logging in?   If you use Hyper-V connect to get to the console of the VM WITHOUT RDP, does it log in fast that way?

Also, another variable is a domain login vs. local login.  Login as a local account, is that significantly different than a domain login?

Let me try to put all the important posts in this one.

1. I don't really know that there is a problem.  These (there are two locations with the exact same issue) are my first Hyper-V's to create myself.  The one created for me at our 3rd location, all logins are fast.  
2. The reason I'm asking the question is I don't want to start installing programs on them and then find out there is a problem.
3. The login time is very fast from the Hyper-v console.  
4. The login time is the same regardless of which user account I use.  I always use Administrator for all servers.
5. The two that have the slow logins were built "before" the fast one.  This is the same at both locations where they are slow.
6. The fast server is a DC
7. When I login to either slow machine, I get the normal "welcome" message.  The fast server, just brings up whatever was on the screen the last time I logged in.
8. All server have same memory and same HD size
All server are on the same subnet and the NICS are configured the same.  NOTE:  As far as I can tell

I think this covers every thing?  Did you look at the two different attachments to see which is correct?

It sounds like on the fast server, you are not fully logging out, but just disconnecting (hence the "whatever was on the screen the last time you logged in")

The slow servers are fully logging you out, and then login process (login scripts, mapping drives, GPO's building the desktop for you, etc) is the 'normal'

On the slow servers, just close your RDP session and then re-establish the session - does it do a full login or just a reconnect of your existing desktop?

Trust me I exit each of them exactly the same way.  That's way this is so puzzling.

Can someone tell me which NIC configuration is correct in my post 39120902

local-10.png is your local adaptor for your host whereby no Hyper-V services are passed through.

local-area-11.png is not a pass through NIC (dedicated) else you would only have the Microsoft Virtual Network Switch Protocol selected. Again this would be configured via Hyper-V Networking.

I don't mean to correct you because I need your help.  However, if you look closer both of those attachments are from the same type of adaptor.  Virtual Network.  Please look again.

Thanks

Then if you have multiple NICS, I would recommend you configure as follows:

Host LAN 1: LAN for host
Host LAN 2: Hyper-V LAN for VMs

In Hyper-V networking, select LAN2 as the 'external' connection and most importantly deselect 'allow management operating system to share this network adaptor'

This way the VM's have a way out of your network and not sharing NICs with the host.

You'll find once configured only the 'Microsoft Virtual Network Switch Protocol' will be ticked.

Reboot your VMs and try again with logging on, etc.

OK, I'll try later today and post the results

I should be able to ping a VM just like any other server, correct?

That's correct.

I've got to be missing something in the NIC set up.  Either on the Host or the VM's.  I was physically there today.  As a test I copied a 203 mb file from one physical server to the Hyper-V Host server.  It took 7 seconds.  I copied the same file from the physical server to one of the VM's.  It took 8 minutes.  

What the Heck am I doing wrong?

I know the host has plenty of memory, hard drive space and a fast processor.  So I've eliminated those items as the cause.  

HELP!

I solved the ping problem.  I didn't have the proper network sharing services turned on.  Speed for coping is better.  Coping from the physical server to the VM is now 60 seconds.  The transfer rate speed of MB per second is less than half of that on the Host.

File and printer sharing needs to be enabled for ping, that's a prerequisite, unless you tinker with firewall rules for ICMP packets.

Considering the VM is running on the hypervisor, most likely on the same disk subsystem on the same partition as the OS, you will experience an overhead and loss of speed when copying files.

Have you tested speeds between VM to VM? Remember its not recommended to use the host for other server roles in a dedicated Hyper-V environment. No AD services, no file share services, etc.

Here are a few best practises for further reading: http://technet.microsoft.com/en-us/magazine/dd744830.aspx

Is your LAN 1 cable and LAN 2 cable connected to the same gigabit switch at the minimum? Are you VMs networking synthetic or legacy? They need to be synthetic.

Do you have AV installed on the host, is it excluding the VHD files? As real time scanning of these files would have an impact on the VM performance and not recommended.

The ping is fixed.  That was my error

The speed from VM to VM is the same as from physical server to VM.  

No AV on the Host.

I moved all LAN cables yesterday to a new switch.  The switch is connected directly to our SonicWall Firewall.  Should they be on different switches?

I'm reading up on on to configure the VM's for Synthetic.  I'll also read the link you posted.  I'll post later.

After all the help I received from EE, I've determined the issue is with the NIC configuration.  The VM's themselves are fast.

I'm reading this article and have no idea where they are referring to.  See last line.

To add and configure a virtual network adapter for a virtual machine:
You can configure virtual machines to use one or more virtual network adapters that connect the virtual machines to internal networks or to external networks after the virtual machines are deployed on a host. To add a virtual network adapter, in the left pane, click Network Adapters, click Network Adapter on the top menu, and then select one of the following:
Emulated network adapter

Emulated network adapters are available on all of the virtualization software platforms and allow virtual machines to be connected to virtual networks. Virtual networks can be connected to other virtual machine network adapters and to host network interface cards.

Synthetic network adapter

Synthetic devices are new with Hyper-V and provide better performance than emulated network adapters. Synthetic network adapters require Virtual Guest Services (VGS) to be installed on the virtual machine. VMM installs VGS for all supported guest operating systems.

In the left pane, click the network adapter, and then, in the results pane, configure the following options:

This is referring to the properties of your VM, when they are off via the Hyper-V Manager. You will be able to see either 'Network Adaptor' (blue symbol) which is synthetic or 'Legacy Network Adaptor' (yellow symbol) which is emulated.

In reply to your questions, no the network cables can be connected to one gigabit switch. I would not advise connecting them to different ports on a Sonicwall unless that's your only choice.

You may want to for troubleshooting disable chimney offloading. This MS article will help you: http://technet.microsoft.com/en-us/library/gg162682(v=ws.10).aspx

Here is a more definitive list of best practises for Server 2008 R2 Hyper-V: http://blogs.technet.com/b/hollis/archive/2012/05/02/windows-server-2008-r2-sp1-hyper-v-best-practices-and-tuning.aspx

Thanks.  I'll do some more reading.  

The VM's aren't connected directly to the SonicWall, the new switch is.

I appreciate your patience.  So based on the attachment, the Adapters are configured correctly?
VM.png

Looks like a synthetic adaptor assigned to the VM, so all good there. I'm assuming LAN 4 is a pass through adaptor for the VMs to your local LAN and not shared with the host?

See earlier post about the tick box.

I believe Lan 4 is a pass through.  But I'll let you confirm.  See attachment
adapters.png

Looks to me you've got a few adaptors configured to share the host LAN connection. Make your setup simple and easy to understand.

Take a look at the attached picture of a production server, serving a similar purpose of hosting multiple VMs which all communicate only via LAN 4 to get onto the network.

LAN 1: Host LAN only
LAN 2: disabled as not used
LAN 3: disabled as not used
LAN 4: VM LAN only

Note in Hyper-V networking, I've selected LAN4 as the 'external' connection and most importantly deselected 'allow management operating system to share this network adaptor'
VM-LAN.PNG

Trust me My LAN connections are very confusing.  In the attached.  The Green are the physical Network Adapters.  The Yellow were created when I installed Hyper-V role.  In your example are all of the LAN's physical?

I'm still not clear on how they should be configured.  Appreciate more help.  

Again, thanks for being helpful and patient.
lan-3.png

@Netflo here is an update. Makes it even more confusing.  I have two VM's that are just members servers.  I added a third VM that is a DC.  Same memory and HD as the other two.  To test the speed I installed Desktop Authority.  It's a program from ScriptLogic that helps me manage all the users Desktops.  It was already installed on one of the VM's.  When I log into it on the VM it was originally installed on it takes 67 seconds to log in.  On the DC VM it takes 13 seconds.
On the original VM it takes 120 seconds to replicate the information to a physical DC.  On the VM DC  it takes 55 seconds.

Any thoughts on this new information?

Did you look at my last attachment with my Network Adapters?

The yellow adaptors look to me like shared NICs, when adding Hyper-V additional adaptors don't get created normally, unless you configure it that way. I would suggest looking at my previous post and clearing them up.

Let's clear up your NIC situation then move on with what's going on.

I've looked at your post multiple times and viewed the image.  If you want me to clean up the NIC I need more instructions.  

Unless I'm losing my mind all the virtual nics in my screen shot were in fact created when I installed Hyper-V Role.  

I think what I should do if take one of my servers that is soon to be removed and add the Hyper-V role to it.  I'll first take a screen shot of the Network Adapter page, then install the role and see what the difference is.  I'll do this tomorrow and send you the screen shots.

OK.  I GOT IT.  yea.  After unchecking the Allow management operating system to share this network adapter, all of the Virtual NIC are gone.  I've now renamed them to make sense.  Example LAN 1 is the first physical port on the server and the NIC that the Host is using.  

Now we can proceed.

That's good. Any improvements to speed or performance of VMs?

unfortunately no.  In my previous post I mentioned that the 3rd VM I added which is a DC, is very fast.  That's the puzzling part.

@Netflo, did you give up?  :-(

Glad to hear you managed to dig deeper and find the solution. Oh no no, I didn't give up. I too have a full time job and other priorities.

I found my own solution