How do I safely get Hyper V VM's out of a failover cluster and onto a standalone hyper v server?

Posted on 2011-10-13
Last Modified: 2012-06-22
I am having problems, major problems, with a new ish windows cluster.  It is only used for Hyper V to make our VM's highly available.

I have 3 nodes each identical in hardware (with the exception of 1 which i have added an additional network card, the plan is to to this to all 3).

The 3 nodes are connected by iSCSI to a Dell md3000i SAN.  each node uses 2 NIC ports for iscsi traffic and i have 2 redundant switches in place.

My problems seem to have started after having added the 3rd node to the cluster and changed the quorum to node majority.  prior to this it was node and disk majority.

Earlier today windows update must have rebooted 1 or more of the nodes (these should obviously not have been set to automatically update).  Now we seem to be stuck in a sort of cycle where 1 or more of the nodes cannot reconnect to the iscsi disks.

It's fairly difficult to try and explain the symptoms.

What i want to do is get rid of the failover clustering role BUT I have to keep the VM's safe.  My plan is to take clustering back to a lab environment and do more testing before putting production servers in there.

i can see 2 options,

1. move the VHD files off to another hyper v server outside the cluster.
2. somehow strip the cluster layer away from the nodes leaving them as normal hyper v servers.

I think i'd prefer option 2 as the hardware is better, to be honest though i'm happy to try any suggestions.

thanks in advance,

Question by:avitman

    Author Comment

    So I decided shortly after asking ths question to start moving machines out of the cluster to a different non clustered Hyper V server.  Shut down each VM, then copy and paste the vhd file to the new server and attach it to a newly created VM.  I did have some problems along the way because the cluster disks kept moving between nodes when they failed over between each other.  I also had to do a small amount of reconfiguring on each newly created/moved VM such as re-assigning an IP address as they all defaulted to DHCP.

    I will continue to troubleshoot the cluster but in a test environment.  The theory behind the technology is great and worth getting right.

    My current thinking is that the problem lies in the iSCSI setup.  I noticed that every time the nodes went down I could RDP into them but the iSCSI disks were missing from disk manager.
    LVL 15

    Expert Comment

    I would troubleshoot the MD3000i and the iSCSi settings on the host servers first to make sure everything is configured properly on them:

    - Use the modular disk configuration utility to configure the servers as oppose to manually setting up the ‘iSCSi initiator’ yourself.  

    - Make sure that you see multiple entries under 'iSCSi Initiator -> 'Favorite Targets' or your iSCSi is not set up properly on the servers. Remove and reconfigure if this is the case

    - Update the MD3000i with the latest firmware if not done already

    - In MD Storage Manager, make sure that you have all 3 servers in one 'Host Group' and the HBAs are entered properly in the represented servers. Also make sure the host type says 'Windows Server 2003/Server 2008 Clustered) in MD Storage Manager -> Host Group -> Host -> Host Type.

    LVL 15

    Accepted Solution

    With respect to the original question, you should use the export function from the Hyper-V manager to export the VM. You can then copy the exported VM to stanalone server and then import the VM. Export option also copies the configuration along with VHD files. You would be spared the reconfiguration part. Just make the VM Network with the same Name on the standalone Hyper-V server.

    Hyper-V Manager is not cluster aware. Failover cluster Manager make VMs Highly available. If you select a VM from cluster Manager and then delete it, you will find that it remove the VM from HA. The VM will become housed on the Hyper-V server where it was located when you removed HA from it (please test it first). But It will still remain on the same storage. Which is why I think this would not be a good option as the disk housing the vhd keep moving/dropping from some of the cluster nodes.

    Author Comment

    I will start the cluster again from scratch just as soon as i get the last of the vm's out.  this is proving to be a challenge as the disk's keep dropping offline!

    There are a couple of config changes I will make on 'attempt 2' including manually setting the speed and duplex on all NIC's.  Also, MS support advised I should have a redundant and dedicated heartbeat network between the nodes, despite other sources (including MS kb articles) claiming that heartbeat networks are a thing of the past in 2008 R2.  I'll add one in any case.

    My feeling is that this is a disk presentation issue, I'm seeing the iscsi disks dissappear quite regularly.

    I have also been advised that Hyper V Server 2008 R2 might be a better option than full blown server 2008 r3 enterprise on the physical cluster nodes.  I'm not sure exactly of the best way in that case to connect the nodes to the SAN, i guess I would have to manually create the connections in iscsi initiator instead of using the dell utility.
    LVL 15

    Expert Comment

    If your cluster was working OK at some point in time then there must be some configuration issue with it.
    I am assuming you are using CSVs for VM storage. Are all online from each node? Are there any in 'redirected access' mode?
    I would advise to look at the following link if there is some problem with CSV.
    Troubleshooting ‘Redirected Access’ on a Cluster Shared Volume (CSV)
    How to fix CSV stuck in redirected mode

    Author Closing Comment

    Thanks this is what I did in the end.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    What Is Threat Intelligence?

    Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

    You might have come across a situation when you have Exchange 2013 server in two different sites (Production and DR). After adding the Database copy in ECP console it displays Database copy status unknown for the DR exchange server. Issue is strange…
    The recent Microsoft changes on update philosophy for Windows pre-10 and their impact on existing WSUS implementations.
    This tutorial will give a short introduction and overview of Backup Exec 2012 and how to navigate and perform basic functions. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as conne…
    How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…

    794 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now