How to : Network resource management/pools

I need to determine a way to deploy network resource limits to individual hosts on a vSwitch, not at the vSwitch level.  I'd rather do this at the ESX level than at the individual host OS-level.

For example, I want hosta/eth0 to be capped to a max throughput of 100mb/s, hosta/eth1 to be uncapped, hostb/eth0 to be capped to 1gb/s etc.

Am unsure if this is possible to do at the host level, in a way similar to resource pooling ram, cpu or disk.

Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Resource Pools is indeed the 'tool' to use to manage resources for varying VMs. They are created on the individual hosts though. See starting on pg. 43 of the Res Mgmt Guide here:

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You can create a VIrtual Machine Network Port Group which is Bandwidth Throttled?
But, resource pools are specifically for CPU/RAM resources. For network resources, you can only use the vSwitch for doing this, and that is also at the host level. You can create VLANs and assign a specific NIC to a vSwitch.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
bandwith throttled Virtual Machine Port Group
sirbunnzAuthor Commented:
Yeah, can't do it at the vSwitch level without extensive reconfiguration.  I dont want to limit the vSwitch throughput, just the virtual ETH device on specific hosts.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Traffic shaping can be performed on the vSwitches (virtual switches), but the vSwitch are after the HOST NIC interfaces, if you wanted to throttle the host interfacaes you would have to throttle on the physical switch, that they are connected to.

But why not group by virtual machine port groups via vSwitches?

two vSwitches with two Virtual Machine portgroups,

1. max throughput of 100mb/s
2. uncapped
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
the NICs on the host are physical not virtual?

Extensive congfiguration, it's just adding a new Virtual Machine Port Group?
Well, just assign a separate NIC (if available) to a 2nd vSwitch and configure bandwidth how you're needing to. Pg. 15 of the Config Guide explains a little bit more on Networking: (good read IMO)
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
What are you trying to achieve, limiting Virtual Machines bandwidth?
sirbunnzAuthor Commented:
Hmm, I think I'm not understanding your comments.

I have 4 physical NICs on the ESX Hosts, 4 virtual switches.  No network redundancy is deployed.  Each switch represents a physically segrated network segment.  One of those segments, vSwitch3 in this instance, is my LAN segment that hosts a lot of servers.  I want to prioritise, or limit, the network usage from specific hosts on my LAN, without limiting in any way the total throughput of the vSwitch.

I'm not running VLANs on any of this and not deploying QOS across the LAN just to deliver this functionality.

Ideally, I want to configure the NICs on the VM's to behave themselves (be thottled or capped), but if I can't do that, I'll have to look at guest OS-Level implementations, which will be complicated.

Is your VM PortGroup things an option?  I"m not sure what that refers to in relation my current environment.  But then, I wasn't all that clear on my current environment I guess.  ;)

Why can they not simply have resource pooling for network resources the same way they have for most other things?  :)

By what I can see, there is a link between a vSwitch and a physical NIC, so the same physical NIC cannot be present on multiple vSwitches?  If there is a way to make this work, then I can probably do the throttling at a vSwitch level and simply deploy my "throttled" hosts to that vSwitch.
VMs use the VM Network Portgroup for their traffic from the virtual NIC they use through the physical NIC of the host. Not all the NIC bandwidth is used solely for VMs...potentially. It's just whatever you have configured for the phys NIC on the vSwitch (could be Mgmt traffic, vKernel traffic for VMotion, etc.). But, it sounds like in your case, vSwitch 3 may be used solely for the VMs? If that is true, then configuring the VM Network portgroup on vSwitch3 would limit the traffic for the VMs that use that vSwitch, and thus the traffic passing through the NIC assigned to vSwitch3. Yes, the same phys NIC canNOT be assigned to multiple vSwitches. But a vSwitch can have multiple phys NICs...for redundancy, etc.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Are you getting hosts confused with guest VMs?

Do you want to throttle the Guest Virtual Machines on the HOST ESXi/ESX Server?

You throttle on the vSwitch internally to the VMs.

There is no overall throttle on the physical NIC in the ESX/ESXi server (HOST).
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You simply create a new Virtual Machine network on ALL your vSwitches.

Edit this Virtual Machine Network, and select the throttle required.

Connect the VMs that you want to be throttled to this NETWORK!

You do not have to configure anything inside the VM.

Throttle is done on the Network Portgroup.

BUT, Network Throttle CAN also be performed on the ACTUAL vSwitch itself.

So this would perform the same function?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
If you want to read more on networking in VMware ESX/ESXi, then I recommend the following:-

I would also recommend reading through the Networking Sections of the following guides to gain a better understanding of Networking in VMware ESX/ESXi.

Pages 13 - 73 Discuss Networking in Detail, including trunks, VLANs, switches, and load balancing

ESXi Configuration Guide ESXi 4.1

Virtual Networking

Virtual Networking Concepts
sirbunnzAuthor Commented:
I may be confusing you with host/guest.

At the ESX HOST Level
- the physical servers have 4 physical NICs, each allocated to a seperate vSwitch.  Each physical NIC is connected to a segregated PHYSICAL network.

At the GUEST OS Level
- I want to rate-limit the network traffic to/from specific hosts
- the guest os in question has 2 virtual nics, connected to 2 different vswitches
- i want to rate-limit the virtual nic connected to vswitch3

The "problem I'm trying to solve" is that the nature of the network is such that some machines ip traffic communicate entirely within the cluster, even when crossing physical network boundaries, without their packets "hitting the wire" and being limited to wirespeed, which esx delivers/sends at full speed.  This causes flow-on affects as my guests try to write data to disks, deliver that data out the physical wire etc.

Basically, an FTP session from a box on vSwitch3 to the DMZ (vSwitch2), has data delivered via IP at a rate faster than its disks can keep up, leading to disk congestion on OTHER requests to access the guests disks.  This issue has cropped up due to the deployment of a collapsed network environment using a virtual firewall, meaning that this traffic now never actually hits a physical wire/speed limit.  In fact, esx reports network speeds of >35,000Kb/s on send/receive to this particular guest in this situation.

Without affecting the other virtual machines, I merely want to "slow down" the rate at which 2 of my hosts can generate IP packets, which obviously isn't something ESX is overly concerned about (SLOWER?  YOU WANT SLOWER?  ARE YOU MAD?!?!?!) and dont seem to make easy.

From what I can see with PortGroups, this again targets the transmission of packets onto the physical wire/NIC, not the virtual NIC.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
create a new virtual network on the vswitch3, apply traffic shaping to this virtual network, and connect guest vm nic to this network?

whats wrong with above?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
yes, i understand what your are stating.....its packet generation in the VM?
sirbunnzAuthor Commented:
Doing that now.

Am not convinced from the documentation that this will actually throttle traffic that DOESNT HIT THE PHYSICAL NIC however and instead heads to another vSwitch.

Thanks for the assist however, will let you know in 15 minutes if it worked or not.
sirbunnzAuthor Commented:
Well, that actually looks to work, despite what the documentation says, excellent.  Thanks for the assist guys!
Awesome...glad you're up and going :)
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
No problems!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.