Trying to Discover the Bottleneck for Storage Vmotion Performance

I'm trying to determine where my bottleneck is as I'm seeing some very poor performance when doing a Storage vMotion migration (migrating VM's to a different host and datastore)

My Vmotion IP subnet is the same as my Management Subnet.( Esxi host management IP subnet on the same subnet as Vmotion)  Is this what is causing the problem?  It is the way this environment was setup before I got here.  Normally I would always separate the Vmotion vlan from the management vlan but that is the way it is here.  

Anyway Is here is my setup

Vcenter 6.5 u1b Build 6816762

Esxi hosts are mixed 5.0, 5.5 and 6.5

Is there a good tool to use to discover the bottleneck?  

I was googling and found iPerf that comes with Esxi 6.5 but ive never used it before.  

Any suggestions?  I really want to improve the storage vmotion performance

JB BlancoSr Systems EngineerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
It's best practice and recommended to have a completely different network for your vMotion/Storage vMotion network, with at least two nics for failover (resilience)  and Jumbo Frames enabled.

This could be dual physical nics (10GBe, 1GBe), and at least 2, 4, 6 to speed up transfer.

VLANs can be used if you have a method to analyse your VLANs and to check overall utilisation of the LINK (and VLANS inside).

BUT what is the issue, because vMotion/Storage vMotion is the real time moving of a VM in real time with no downtime, so do you just want to make it quicker.....

but if it succeeds why?

also bottlenecks in Storage, Read and Write can be an issue ?

Is it currently affecting production, and how often are your Storage vMotioning...

we use SDRS clusters, and VMs are moved round the datastores and we just let them move, we don't watch it!!!!

You can also look at your current performance using esxtop and vCenter Server.
JB BlancoSr Systems EngineerAuthor Commented:
We are not Svmotioning very often at all, good point.  I just noticed it since I am having to migrate some and I just noticed how painfully slow it is.  Yes as I mentioned I don't like the way they have it setup with Vmotion and management on the same vlan.  But that's the way it is and I don't think we can change it.  

But is there a tool of some sort I can use to discover the bottleneck more efficiently?  you said esxtop.  Is there a list of commands?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Yes as I mentioned I don't like the way they have it setup with Vmotion and management on the same vlan.  But that's the way it is and I don't think we can change it.  

Not recommended.

If you cannot change it tough! that's how it's going to be.... what differences it makes to complete not sure because service is still up! or should be!

1. Enable Jumbo Frames.
2. Dedicated two nics, more nics the better for concurrent vMotions. 10Gbe where possible.
3. Bottlenecks - not often LAN
4. Bottlenecks - CPU, Storage

Check \var\log

see this article I wrote (also valid for 6.x)

HOW TO:  Performance Monitor vSphere 4.x or 5.0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.