How should I configure an identical server for failover purposes?

OK, I'll try to keep things as simple as possible.
We have one domain.
We have a virtual host (ServerA) that has 8 virtual machines stored on it.
We have an identical host (ServerB) that has copies of all 8 virtual machines.
The virtual hosts themselves aren't accessed directly. The virtual machines are what the users access.
The 8 VMs on ServerA have static IP addresses and certain names.
The 8 VMs on ServerB which are copies of the 8 VMs from ServerA have dynamic IPs and have an underscore in front of their name.  This allows all 16 VMs to be on our network at the same time.
In the event ServerA was to fail, I want to be able to:
- remove the underscore from each VM on ServerB
- give each VM on ServerB the static IP their brother's had on ServerA
- reboot each VM and have them now be the primary, active VMs

I'm trying to minimize downtime.  I know there are solutions out there that can do all of this automatically like VMware HA but these require a SAN or vSAN which we don't have money for.
That being said, I ran into an issue when running the test below:
- shut down VM1 on ServerA
- assign _VM1 on ServerB the static IP address VM1 had on ServerA
- rename _VM1 to VM1 and reboot

When the "new" VM1 comes back up, we can't login because of a domain trust issue. This requires us to login locally, remove the VM from the domain, reboot, add it back to the domain and reboot once more.
This adds an additional XX minutes which I'd like to see if we can avoid.  
I'm assuming this is because the original VM1 was already on the domain so Active Directory doesn't like that there is another VM1 trying to login to the domain.
Can I do something to prevent this like remove VM1 from AD in the event of a failure?
Any help would be appreciated.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Why don't you just replicate VMs  (copy, restore from Host A)

from Host A to Host B....

and if you need to turn on VMs, in the event of Host A failure, just power them up....

with no modifcations to IP Address, or Inventory name?

You will always have trust issues in AD, it's based on the machine Domain Account, not the IP Address.

Why do you want ALL VMs to be on the network at the same time, this is the issue!
homerslmpsonAuthor Commented:
There are network shares on most of the VMs.
I'm not sure what method of copying the VMs you are referring to but I'd think no matter the method, data wouldn't be 100% in sync.  If this is the case, a copy/restore would be missing data.
My plan as of now is to use Robocopy's monitor parameter on each VM to keep all of the shared folders in sync. While I realize this isn't syncing data real time, a value of 1 seems to keep things relatively up to date.
I suppose I could setup the hosts as follows which would reduce the restore time by 50% in the event of a failure:

ServerA:       ServerB:
VM1            _VM1
VM2            _VM2
VM3            _VM3
VM4            _VM4
_VM5           VM5
_VM6           VM6
_VM7           VM7
_VM8           VM8

Open in new window

This would increase the chance of a server failure by 100% but would reduce the time it takes to fix the issue because 4 of the 8 VMs would already be in the right spot.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Purchase a copy of Veeam Backup and Replication, keep your single VMs on Host A, and replicate to Host B.

or purchase a Backup and Restore product, and restore VMs to Host B, the next morning.

What you are currently doing is prone to failure because the Windows machines are part of the Domain.

So you will have to have unique machine names and different IP addresses, and then perform "the swap" at Host down.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

homerslmpsonAuthor Commented:
We already have Veeam. We use it for backing up our VMs to disk for DR purposes.
I looked at it recently as you weren't the first person to mention Veeam.  I don't recall seeing anything in that software that looked helpful.
Does replication happen real time or is it something that is scheduled daily/weekly?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Replication is user configurable, you create a Veeam Replication, job and VMs can replicate every 15 minutes if required to a host.
homerslmpsonAuthor Commented:
In the scenario I provided, am I supposed to make the entire array on ServerB a Veeam backup repository?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You can replicate

VMs from Host A to Host B

VMs from Host B to Host A


all VMs from Host A to Host B (standby server and VMs)

via Veeam Backup and Replication Jobs.

there is then no requirement to change machine names, IP Addresses, and in the event of Host failure, just turn on VMs on the Host.

The VMs, if set to 15 minutes, will be 15 minutes old, which is pretty good for DR.

If you require better recovery than 15 minutes you will need to look at different technology.

There is also a benefit, because once you have fixed Host A (failed host), Veeam can rollback......replicate the other way. (up to you how you work it!)
homerslmpsonAuthor Commented:
That sounds like a great idea but I could use a hand implementing it.
I like the idea of having all of the VMs on ServerA and replicating every 15 minutes to ServerB.
In Veeam you need to setup a backup repository.
I don't know how this should be done. Clearly we want the VMs to be stored on ServerB but I don't know how to create a repository/share using vSphere.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Create a Replication Job
homerslmpsonAuthor Commented:
I did create a replication job. It says you need to make a repository first.
I went through the steps to make one and my options were somewhat limited.
I could use local storage on the machine Veeam is installed on, a Windows network share, etc.
I don't know how to tell it to place the images on the 4TB of disk space on ServerB.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Local Storage, Windows Network Share is supported as a repo.

There is some confusion about the repository, this is a location where the metafiles for the backup (job info etc are stored)

later in the job, you'll get to select the datastore on the ESXI host!

where the replica files are stored (e.g. the vmdk!)
homerslmpsonAuthor Commented:
OK I think I understand now.
I may have been looking at BACKUP jobs not REPLICATION jobs.
I'm now running a test replication job but am getting terrible transfer speeds.
I'm replicating from one virtual host to the other and am getting a speed of 11 MB/s.
The bottleneck showing in Veeam is the TARGET.  
Any ideas why this is happening?
As of right now there is nothing on the 2nd virtual host besides the ESXi OS itself.
And thank you for your help!
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Speed of disk, network speed.
homerslmpsonAuthor Commented:
They are both connected to the same 1Gbps switch.
They are both running a RAID10 w/ 8 x 1TB SAS drives.
I know they aren't SSD or anything but jeez there's nothing even on them yet.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
We you getting better Backup speeds, this is probably worth a new question, on Veeam specific configuration.
homerslmpsonAuthor Commented:
Looks like this has to do with Veeam needing a proxy server on the target server.
Veeam itself acts as a proxy server on the host side but one is needed on the target side for optimal performance.
I was on the phone with Veeam for 90 minutes talking to one of their techs and after everything is all said and done, it would likely be a BAD idea to setup replication every XX minutes due to the performance hit the datastore would take.
Looks like I may need to figure something else; possibly some type of hybrid system.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
That depends on how good the performance of your datastore is.

and how much downtime you can afford to lose....

60 minutes...

otherwise products like DoubleTake HA, are better at this....because data is replicated at the VM level, not the entire VM...
homerslmpsonAuthor Commented:
The goal was to try and avoid purchasing additional software. The initial software purchase is one thing but you need to have active maintenance in order to get support which in some cases is quite costly and continues indefinitely.
I'm working with Veeam on the speed issue and hopefully they can help me resolve it.
Thanks for you help.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.