vSphere Replication is slow replicating VM

vSphere Replication is very slow. I am trying to replicate VMs as we are trying to move out data enter to a virtual data centre (we will be renting computing and storage). My hosts are ESXi 5.0, vCenter 5.1 and vSphere Replication software that works with ESXi 5.0 and vCenter 5.1.  I am trying to migrate VMs from a cluster to a stand-alone host which has its own storage and two data centres are connected via a 300 Mbps connection.  I am averaging 512 Kbps only for the replication.
LVL 25
Mohammed KhawajaManager - Infrastructure: Information TechnologyAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

systechadminConsultantCommented:
Did you checked at the network end. there are lot factors causing the issue. Network, Storage IOps, Read write latency on the disk. Kindly check all these parameters
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
if you manually copy a file, to your destination, how long does it take for a 40GB VM ?
0
Mohammed KhawajaManager - Infrastructure:  Information TechnologyAuthor Commented:
I cannot do a file copy test as I do not have a VM on the other ESXi host.  I will try this later on.
With respect to WAN, it has been tested and latency is very low.  SAN storage is used at both ends, source SAN is IBM V7000 and target SAN is NetApp FAS2500.  I have checked the IBM SAN and there is nothing out of ordinary.  Read latency on the IBM is 7 ms and write latency is 0 ms.

Source ESXi is a cluster using vSphere Enterprise Plus licenses and each host is under utilized (less than 10% CPU and 67% RAM).  Target ESXi has 256 GB RAM and there are no VMs on it.

With respect to networking, hosts are IBM HS22V blades in an IBM BladeCenter-H chassis and each host has 4 physical 1 Gbps NICs (Nic1= Management and failover for vMotion and backup, Nic2 = vMotion and backup, failover for Management, Nic3 and Nic4 are configured as a distributed switch for virtual machines.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I am more interested, in the speed of your WAN link.
0
Mohammed KhawajaManager - Infrastructure:  Information TechnologyAuthor Commented:
Performed tests between the two data centers and I am achieving 200+mbps transfer rate.  Added a host to the ESXi in the remote site and performed FTP transfer test between two VMs (one at target and one at source) and I am getting 200+mbps transfer speed.  Tried replication again and I am seeing transfer speeds of less than 5mbps.

Networking is not issue, resources on the hosts and SANs are not an issue.  I am beginning to think that SSL may have something to do with this but I am not sure as to how it could be turned off.
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
is this the first seed ?
0
Mohammed KhawajaManager - Infrastructure:  Information TechnologyAuthor Commented:
It is the first seed and also note that the next day, 1.34 GB of delta changes where written in 39 minutes which again comes to 4.x Mbps.
0
Mohammed KhawajaManager - Infrastructure:  Information TechnologyAuthor Commented:
Update:

vSphere Replication Appliance cannot be tweaked to work any faster and is designed to replicate many VMs simultaneously with similar throughput.  In a nutshell, I could replicate approximately 40 VMs concurrently each averaging around 4.x Mbps.  

If hosts are ESXi 5.0 then you could tweak the host as per link below to get approximately twice the speed and in this scenario I could replicate about 20 VMs concurrently.
http://blogs.vmware.com/vsphere/2012/06/increasing-vr-bandwidth.html

Performance improvements have been made in ESXi 5.5 where throughput is approximately 3 times the speed of ESXi 5.0 as per link below:

https://blogs.vmware.com/vsphere/2013/10/vr-5-5-performance-findings.html

Recommended way for replication large VMs is to send VMDK files to the destination site, upload them to the datastore and then performance delta sync afterwards.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Mohammed KhawajaManager - Infrastructure:  Information TechnologyAuthor Commented:
No real good suggestion or solution was provided.  Closing the request as there is not much that can be done.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Virtualization

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.