VMware ESX nic teaming, vmotion, console, vm
i have 2 - ESX 4 hosts each with 6 nic ports available, 2 builtin 2 on dual port nics . Running VSphere. New setup. iScsi to hp1510i. iScsci connected just fine on the first host. I need some guidance on proper layout of switches/port groups/nics. Want load balancing/failover/perform
This is my assumption:
HOST1
vmnic0 (builtin)= console
vmnic1 (builtin)=vm network
vmnic2 (card1)=iscsi
vmnic3 (card1)=vm network
vmnic4 (card2)=iscsi
vmnic5 (card2)=vmotion
Any suggestions? Should i get an additional 4 port nic so i can have redundant console vmnics and maybe an additional 2 iscsi connections since the hp1510i does have dual controllers? Is it possible to have 2 vmnic's for vmotion? should vlan vmotion or separate it from iscsi/vm networks?
This is my assumption:
HOST1
vmnic0 (builtin)= console
vmnic1 (builtin)=vm network
vmnic2 (card1)=iscsi
vmnic3 (card1)=vm network
vmnic4 (card2)=iscsi
vmnic5 (card2)=vmotion
Any suggestions? Should i get an additional 4 port nic so i can have redundant console vmnics and maybe an additional 2 iscsi connections since the hp1510i does have dual controllers? Is it possible to have 2 vmnic's for vmotion? should vlan vmotion or separate it from iscsi/vm networks?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
dnilson, I suggest that you save your screen shots as JPGs or PNGs and then upload them to reduce their sizes. BMPs are comparatively larger in size.
I know .... but far more portable....
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Service Console on the vSwitch2 is only needed if you are using a iSCSI Software Initiator, if you are using a iSCSI Hardware Initiator or TOE Adapter, you do not need a second Service Console.
paulsolov - Is it possible to have 2 VMKernel Ports configured for vMotion ?
paulsolov - Is it possible to have 2 VMKernel Ports configured for vMotion ?
Not sure if its a SAN dependent issue or not but with te iSCSI SAns we use (DellMD300i), its not possble to configure them without a vKernel o nthe iSCSI vSwitch - ESX complains that its missing, and wont accept it on the console port instead.
Currently configured with a vKernel on bo iSCSI ports, But Physical NICS are configured in failover, not teaming mode, which gets back to arunaju's question above. My config never has both vKernels active at the same time.
Currently configured with a vKernel on bo iSCSI ports, But Physical NICS are configured in failover, not teaming mode, which gets back to arunaju's question above. My config never has both vKernels active at the same time.
arunraju - I assumed iscsi initiators since each the toecards cost about 40% of the MSA1510 cost and if my memory serves me correctly it comes with 7.2K SATA drives which means that the hard drive I/O will be a factor way before iSCSI throughput.
Not sure if you can do vmotion on two vswitches but I'll try it out, what I was trying to say is that when I don't have enough ports I configure the console and vmkernel port for vmotion on the same vswith so that we have physical nic redundancy. If customer has Cisco gear configuring Etherchannel for aggregate throughput also helps out.
Not sure if you can do vmotion on two vswitches but I'll try it out, what I was trying to say is that when I don't have enough ports I configure the console and vmkernel port for vmotion on the same vswith so that we have physical nic redundancy. If customer has Cisco gear configuring Etherchannel for aggregate throughput also helps out.
ASKER
guys, thx for all your quick feeback
The VMnetwork represents vm's, initially it will only be about 6 on each host. I am using iscsi software initiators. The msa1510 has scsi hd's not sata drives and i do have separate switches with differnet subnets for iscsi traffic. No cisco gear. I like the idea of using 2 vmnics for both service console & vmotion, at least until i can get more physical nics. I believe i can do teaming/load balancing with my equipment. Supposedly this new firmware version for the HP1510 supports multipathing with esx3, not sure if it will work with esx4.
paulsolov: thanks for your sample and recommendations
dnilson: thanks for the screenshoot. I see that you left you lan vm's with the service console. Do they HAVE to be together? if i create a new switch for Virtual machines, does it need a service console as well? I ask because your DMZ group doesn't have a service console.
I also do see a service console port for your iscsi, so i will do that as well.
Another question is performance: how many vmnic's are recommended for iscsi and LAN? I will only have about 6-10 vm's on each host and they will be clustered. Have 16gb of ram on each host as of now and dual proc.
The VMnetwork represents vm's, initially it will only be about 6 on each host. I am using iscsi software initiators. The msa1510 has scsi hd's not sata drives and i do have separate switches with differnet subnets for iscsi traffic. No cisco gear. I like the idea of using 2 vmnics for both service console & vmotion, at least until i can get more physical nics. I believe i can do teaming/load balancing with my equipment. Supposedly this new firmware version for the HP1510 supports multipathing with esx3, not sure if it will work with esx4.
paulsolov: thanks for your sample and recommendations
dnilson: thanks for the screenshoot. I see that you left you lan vm's with the service console. Do they HAVE to be together? if i create a new switch for Virtual machines, does it need a service console as well? I ask because your DMZ group doesn't have a service console.
I also do see a service console port for your iscsi, so i will do that as well.
Another question is performance: how many vmnic's are recommended for iscsi and LAN? I will only have about 6-10 vm's on each host and they will be clustered. Have 16gb of ram on each host as of now and dual proc.
You don't need to have service console on a vswitch with a VM Network. When you configure the service console for iSCSI vswitch you don't need to put the defautl gateway (it will ask if you want to do this). It is not needed
Performance considerations should be calculated in such a way that Servers which are I/O Intensive literally do not plague a single LUN.
Balance your workloads across multiple LUNs.
Balance your workloads across multiple LUNs.
Funny you should ask.
When I picked up this project from the last guy, he HAD a Service Console on the DMZ vSwitch, but I felt the potential exposure of being able to login to the management console from anywhere on the Internet just too much of of a liability to leave it there -- so no the Service console is places where you want to do management from, or where the external stroage needs to communicate back to the managemen console, and no where else.
I would use 2 vmnics/physical nics for iSCSI,each routed to a seperate gigabit ethernet switch as suggested above, otherwise the switch can become a single point of failure. To avoid making that a big budget item, make sure you get enough ports to be able to create seperate VLANs for the various subnets, that way instead of one switch for iSCSI, and one for network, you use two switches for each subnet and get the hardware redundancy.
You dont need to extend the VLAN out to the hsts or other network servers, just trunk them togeher at the switch and everything you plug into VLAN 1 on switch 1 will talk to VLAN 1 on the second switch, and none of the hosts, guests, or servers need to be configured for VLAN
When I picked up this project from the last guy, he HAD a Service Console on the DMZ vSwitch, but I felt the potential exposure of being able to login to the management console from anywhere on the Internet just too much of of a liability to leave it there -- so no the Service console is places where you want to do management from, or where the external stroage needs to communicate back to the managemen console, and no where else.
I would use 2 vmnics/physical nics for iSCSI,each routed to a seperate gigabit ethernet switch as suggested above, otherwise the switch can become a single point of failure. To avoid making that a big budget item, make sure you get enough ports to be able to create seperate VLANs for the various subnets, that way instead of one switch for iSCSI, and one for network, you use two switches for each subnet and get the hardware redundancy.
You dont need to extend the VLAN out to the hsts or other network servers, just trunk them togeher at the switch and everything you plug into VLAN 1 on switch 1 will talk to VLAN 1 on the second switch, and none of the hosts, guests, or servers need to be configured for VLAN
ASKER
i'll give all the suggestions a try today and test this weekend. much appreciated.
ASKER
quick question, the service consoles for iscsi have to be on which subnet?
say the vSwtich0 service console is at 10.10.10.1
vSwitch1 iscsi vkernel port is at 172.16.1.1
If i add the service console to vSwitch1, what IP and GW subnet would i give it? i assume it would have to be on the same subnet as the iscsi but i'm not sure. Do all service consoles have to communicate?
say the vSwtich0 service console is at 10.10.10.1
vSwitch1 iscsi vkernel port is at 172.16.1.1
If i add the service console to vSwitch1, what IP and GW subnet would i give it? i assume it would have to be on the same subnet as the iscsi but i'm not sure. Do all service consoles have to communicate?
The IP Address would be on the same subnet as your vmkernel 172.16.1.x. If it asks you for to setup the default gateway decline it.
Just re-read the thread. If you're using ESX4 you do not need a Service Console on the iSCSI vswitch, it is only needed with ESX3.5 but it's not a bad idea to have it just incase the main service console become unavailable you can get into your host through the second one
I generally recommend that the second Service Console be configured with a different Subnet/vSwitch instead of teaming a NIC to the first vSwitch0 (vswif0).
I think that its not possible to have redundant vMotion NICs. Max no of vMotion NICs enabled can only be 1.