Hyper-V R2 Live Migration Fails after network interfaces confgurations became 'scrambled'
Posted on 2010-09-16
I am running a two server failover cluster on Windows Server 2008R2 datacenter edition to support HAVM's. I am running 2 Dell R710's connected to a Dell MD3000I ISCSI Chassis. The ISCSI is configured in multipath and the ISCSI console shows the connectivity is good.
Each host server has 8 network interfaces. 3 of which are dedicated for Hyper-V, 1 is shared between Hyper-V and host management (as a backup), 1 is configured for dedicated host management, two configured for the ISCSI connectivity and the last for the failover cluster.
Here is a logical breakdown of the network adaptors, they are the same on both servers (OB= Onboard -Broadcom / AI = Add In card - Intel)
OBE1 - ISCSI01 - ISCSI Traffic only - IPV4 only - dedicated (unique) subnet 01
OBE2 - failover Cluster - cluster traffic only - IPV4 - MS Client and File & Print services - Cluster communications - No client communications - dedicated (unique) subnet 02
OBE3 - Main network lan - dedicated interface - IPV4 - IPV6 - MS Client - F&P
OBE4 - HVNet01 - dedicated to Hyper-V virtual guest networking - External no OS management
AI01 - ISCSI02 - ISCSI traffic only - IPV4 only - dedicated (unique) subnet 03
AI02 - HVNet02 - dedicated to Hyper-V virtual guest networking - External no OS management
AI03 - HVNet03 - dedicated to Hyper-V virtual guest networking - External no OS management
AI04 - HVNet02 / backup host management - Hyper-V virtual guest networking - with Host management - IPV4 - MS Client - F&P
I was able to Live Migrate a week ago, all was working. Sometime in the past week my network cards on each host became ‘scrambled’ What I mean by that is the configuration for OBE2 ended up on AI03, the configuration for AI03 ended up on Ai02, the interface name of OBE1 (a broadcom interface) was now linked to AI04 (Intel add in card) … each of the host servers were scrambled differently.
After a few hours of painstakingly correcting the networking configurations (I had to pull the network cable from each port in turn so I could track them down), then going back and correcting the Hyper-V networking links under Hyper-V manager I am here.
When I attempt to live migrate a server it fails. It appears the configuration files move but the CSV doesn’t. I can migrate the CSV without issue, but the Hyper-V guest does not migrate.
The validation says the configuration is suitable for clustering on both servers.
I am getting a barrage of event ID’s 1196, 1579, 5120, 5142.
I cannot seem to track this down, any help would be appreciated. And when this gets fixed how do we prevent it again, it seems odd the network interface card could get scrambled in such a way.
The error received when trying to Migrate using VMM is:
Virtual machine SP01 could not be live migrated to virtual machine host VHOST01 using this cluster configuration.
(Unspecified error (0x80004005))
Check the cluster configuration and then try the operation again.
The error received when trying to migrate using failover Cluster Manager is:
Migration Attempt failed
'SCVMM CPC-SP01' live migration did not succeed at the destination.
Live migration did not succeed: The operation was canceled.
As I said I can see it try to move the VM configuration but it never takes the CSV offline or tries to move it.