Solved

VMware ESX nic teaming, vmotion, console, vm

Posted on 2009-07-09
17
2,463 Views
Last Modified: 2012-05-07
i have 2 - ESX 4 hosts each with 6 nic ports available, 2 builtin 2 on dual port nics . Running VSphere. New setup. iScsi to hp1510i. iScsci connected just fine on the first host. I need some guidance on proper layout of switches/port groups/nics. Want load balancing/failover/performance. My hp1510i has 2 controllers running new v2.02 firware which made them run in active/active.
This is my assumption:
HOST1
vmnic0 (builtin)= console
vmnic1 (builtin)=vm network
vmnic2 (card1)=iscsi
vmnic3 (card1)=vm network
vmnic4 (card2)=iscsi
vmnic5 (card2)=vmotion

Any suggestions? Should i get an additional 4 port nic so i can have redundant console vmnics and maybe an additional 2 iscsi connections since the hp1510i does have dual controllers? Is it possible to have 2 vmnic's for vmotion? should vlan vmotion or separate it from iscsi/vm networks?
0
Comment
Question by:dguillen
  • 5
  • 5
  • 4
  • +1
17 Comments
 
LVL 19

Expert Comment

by:vmwarun - Arun
ID: 24819331
Make sure that you have redundant iSCSI Connections to different Switches else that might be a Single Point of Failure.

I generally recommend that the second Service Console be configured with a different Subnet/vSwitch instead of teaming a NIC to the first vSwitch0 (vswif0).

I think that its not possible to have redundant vMotion NICs. Max no of vMotion NICs enabled can only be 1.
0
 
LVL 10

Assisted Solution

by:dnilson
dnilson earned 100 total points
ID: 24819449
Here is a portion of a diagram showing the necessary vSwitch, Service consoles and vKernels

Every Physical NIC needs a vSwitch
Each vSwitch can have, but doesnt require a seperate subnet
The managment (Console) Physical NIC/vSwitch needs a Service Console
The two iSCSI NICs should be on seperate vSwitches, subnets and each needs a vKernel.

Not sure what the VMnetwork represts in you setup, but it could have a service console, making the console NIC unecessary, unless they are seperated for security reasons.
esx-netmap.bmp
0
 
LVL 10

Assisted Solution

by:dnilson
dnilson earned 100 total points
ID: 24819462
Note in the diagran that vSwitch0 has two physical NICs, these are configured in a failover config, but if the hardware supports it, could be in a taming configuration.  We dont because the Polycom VOIP phones wee use react very badly to a teamed Broadcom NIC, and frankly we havent needed the extra performance, just the security.

This scheme, wheter failover or teaming can be replicated to each vSwitch depending o nyour own plan for HA.
0
 
LVL 19

Expert Comment

by:vmwarun - Arun
ID: 24819475
dnilson, I suggest that you save your screen shots as JPGs or PNGs and then upload them to reduce their sizes. BMPs are comparatively larger in size.
0
 
LVL 10

Expert Comment

by:dnilson
ID: 24819486
I know .... but far more portable....
0
 
LVL 42

Accepted Solution

by:
paulsolov earned 150 total points
ID: 24819584
Here's how I've done it with your config and the MSA1510i.  If you have two controllers you may not be able to do multipath (at least in the prior fimware but that's ok in most situations)

Vswitch 0 -Service Console and Vmotion (Vmkerne).  Combining the two SC and Vmotion gives you more redundancy since you have two ports on different nics.  If you had more ports you could separate them
vmnic 0
vmnic2


vswitch 1- VMNetwork
vmnic1
vmnic4


vswitch 2 ISCSI.-  I would create a separate VLAN or put it on a separate switch along with the SAN to segrate data.  You'll also need to install service console on this vswitch so that it can talk to the main SC
vmnic3
vmnic5


0
 
LVL 19

Expert Comment

by:vmwarun - Arun
ID: 24819627
Service Console on the vSwitch2 is only needed if you are using a iSCSI Software Initiator, if you are using a iSCSI Hardware Initiator or TOE Adapter, you do not need a second Service Console.

paulsolov - Is it possible to have 2 VMKernel Ports configured for vMotion ?
0
 
LVL 10

Expert Comment

by:dnilson
ID: 24819643
Not sure if its a SAN dependent issue or not but with te iSCSI SAns we use (DellMD300i), its not possble to configure them without a vKernel o nthe iSCSI vSwitch - ESX complains that its missing, and wont accept it on the console port instead.

Currently configured with a vKernel on bo iSCSI ports, But Physical NICS are configured in failover, not teaming mode, which gets back to arunaju's question above.  My config never has both vKernels active at the same time.
0
VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

 
LVL 42

Expert Comment

by:paulsolov
ID: 24819730
arunraju - I assumed iscsi initiators since each the toecards cost about 40% of the MSA1510 cost and if my memory serves me correctly it comes with 7.2K SATA drives which means that the hard drive I/O will be a factor way before iSCSI throughput.  

Not sure if you can do vmotion on two vswitches but I'll try it out, what I was trying to say is that when I don't have enough ports I configure the console and vmkernel port for vmotion on the same vswith so that we have physical nic redundancy.  If customer has Cisco gear configuring Etherchannel for aggregate throughput also helps out.
0
 
LVL 1

Author Comment

by:dguillen
ID: 24824244
guys, thx for all your quick feeback
The VMnetwork represents vm's, initially it will only be about 6 on each host. I am using iscsi software initiators. The msa1510 has scsi hd's not sata drives and i do have separate switches with differnet subnets for iscsi traffic. No cisco gear. I like the idea of using 2 vmnics for both service console & vmotion, at least until i can get more physical nics. I believe i can do teaming/load balancing with my equipment. Supposedly this new firmware version for the HP1510 supports multipathing with esx3, not sure if it will work with esx4.
paulsolov: thanks for your sample and recommendations
dnilson: thanks for the screenshoot. I see that you left you lan vm's with the service console. Do they HAVE to be together? if i create a new switch for Virtual machines, does it need a service console as well? I ask because your DMZ group doesn't have a service console.
I also do see a service console port for your iscsi, so i will do that as well.

Another question is performance: how many vmnic's are recommended for iscsi and LAN? I will only have about 6-10 vm's on each host and they will be clustered. Have 16gb of ram on each host as of now and dual proc.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 24825381
You don't need to have service console on a vswitch with a VM Network.  When you configure the service console for iSCSI vswitch you don't need to put the defautl gateway (it will ask if you want to do this).  It is not needed
0
 
LVL 19

Expert Comment

by:vmwarun - Arun
ID: 24825675
Performance considerations should be calculated in such a way that Servers which are I/O Intensive literally do not plague a single LUN.

Balance your workloads across multiple LUNs.
0
 
LVL 10

Expert Comment

by:dnilson
ID: 24826018
Funny you should ask.

When I picked up this project from the last guy, he HAD a Service Console on the DMZ vSwitch, but I felt the potential exposure of being able to login to the management console from anywhere on the Internet just too much of of a liability to leave it there -- so no the Service console is places where you want to do management from, or where the external stroage needs to communicate back to the managemen console, and no where else.

I would use 2 vmnics/physical nics for iSCSI,each routed to a seperate gigabit ethernet switch as suggested above, otherwise the switch can become a single point of failure.  To avoid making that a big budget item, make sure you get enough ports to be able to create seperate VLANs for the various subnets, that way instead of one switch for iSCSI, and one for network, you use two switches for each subnet and get the hardware redundancy.

You dont need to extend the VLAN out to the hsts or other network servers, just trunk them togeher at the switch and everything you plug into VLAN 1 on switch 1 will talk to VLAN 1 on the second switch, and none of the hosts, guests, or servers need to be configured for VLAN
0
 
LVL 1

Author Comment

by:dguillen
ID: 24826323
i'll give all the suggestions a try today and test this weekend. much appreciated.
0
 
LVL 1

Author Comment

by:dguillen
ID: 24842655
quick question, the service consoles for iscsi have to be on which subnet?
say the vSwtich0 service console is at 10.10.10.1
vSwitch1 iscsi vkernel port is at 172.16.1.1
If i add the service console to vSwitch1, what IP and GW subnet would i give it? i assume it would have to be on the same subnet as the iscsi but i'm not sure. Do all service consoles have to communicate?
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 24843518
The IP Address would be on the same subnet as your vmkernel 172.16.1.x.  If it asks you for to setup the default gateway decline it.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 25576096
Just re-read the thread.  If you're using ESX4 you do not need a Service Console on the iSCSI vswitch, it is only needed with ESX3.5 but it's not a bad idea to have it just incase the main service console become unavailable you can get into your host through the second one
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

When we have a dead host and we lose all connections to the ESXi, and we need to find a way to move all VMs from that dead ESXi host.
In this article, I show you step by step with screenshots to assist you - HOW TO: Deploy and Install the VMware vCenter Server Appliance 6.5 (VCSA 6.5), with some helpful tips along the way.
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…
In this video tutorial I show you the main steps to install and configure  a VMware ESXi6.0 server. The video has my comments as text on the screen and you can pause anytime when needed. Hope this will be helpful. Verify that your hardware and BIO…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now