Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Vmware Physical Network and Storage Connectivity

Posted on 2014-03-29
24
Medium Priority
?
603 Views
Last Modified: 2014-03-31
in most of Vmware environments when they bring an ESX host to the network, they use Network and storage redundancy.
I mean they connect 2 fiber channel adapters from ESX host to 2 separate fiber Channel Network Switches
and they connect 2 finder channel adapters from ESX host to 2 separate fiber channel Storage Switches.

I would like to know what is the best practice for Network and Storage physical connection while  taking into account the need to separate Management Network (Which I guess is the traffic that goes between Vcenter and ESX hosts), vMotion, sVmotion, and Management traffic ( I guess it is for VMHA) and VM Network I guess (traffic between Virtual Machines).

Thank you
0
Comment
Question by:jskfan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 4
  • 3
  • +2
24 Comments
 
LVL 25

Expert Comment

by:Mohammed Khawaja
ID: 39963912
What you need for networking is plenty of network cards.  As an example with 4 NICs, I would make NIC 1 for management and backup for vmotion and backup.  NIC 2 would be the reverse of 1.  I would team NICs 3 and 4 and I have redundancy.  If I had 8 NICs then I could have dedicated NICs for management, vmotion, backups and vm traffic.   As always have multiple storage controllers and with fiber channel cards, most SANs support active paths.  If you are using iSCSI then you could have multiple NICs as well as teamed NICs.
0
 

Author Comment

by:jskfan
ID: 39964028
if I understand VM Network (Virtual machine port Group) is the one that has the heavy traffic. Correct ?
if so then I believe we need to dedicate VM Network  to the 2 Fiber channel physical Nics

for vmotion we can add ethernet NIC
for Management Network ( Vcenter to ESX communication) we can add ethernet adapter
for Management Traffic( VMHA) we can add ethernet adapter
0
 
LVL 13

Assisted Solution

by:Abhilash
Abhilash earned 500 total points
ID: 39964581
You will not need two cards for management. One NIC is enough. You will need one for vMotion and One or Two for VM traffic depending on the kind of traffic you generate and network redundancy you choose.
We run out environment with 4NIC and 2HBA's on each server. That gives us redundancy for both storage and Network.
Of course we also have redundant switches and also redundancy is take care of at port level on the storage array.
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 
LVL 62

Assisted Solution

by:gheist
gheist earned 500 total points
ID: 39964755
One of most important considerations in vmware network design is complete isolation of vmware service network and user networks.
You sail you want redundancy - so 2 adapters for user networks and 2 for system
In system one is primary for vmotion another primary for storage, and 1 control network on each.

Since you have separate storage redundancy setup you can allocate 1 adapter for system uses (as it loses control connection it keeps wortking, or vmotion is cancelled.)
0
 

Author Comment

by:jskfan
ID: 39964855
I guess we are not in the same page regarding Terminology, When I say:

**VM Network = I mean Virtual Machine port Groups, to my understanding this handles the communication between Virtual Machines through ESX hosts and Vcenter with ESX hosts.
I believe the 2 fiber channels that connect ESX host to the Network Switch will be enough to do the job.

***Vmotion = I mean the moving of the Virtual machines between ESX Hosts (the body of the VM stays in the same datastore, just RAM/CPU/NIC traffic will switch to another ESX host). I thought this will be handled by the 2 Fiber Channels that connect ESX Host to Fiber Channel Storage Switch.

**Management Traffic= I mean VMHA, to my understanding this handles the heartbeat between Master ESX host and other ESX Hosts as well as VM machines.
We can either add an Ethernet adapter to dedicate VMHA traffic to , or just make VMHA traffic goes through Network Adapters already assigned for vMotion or VM Network.


I could be wrong in the Terminology and Role of each component, if so,  please correct.

Thanks
0
 

Author Comment

by:jskfan
ID: 39964862
please, just to make it clear, when you mention any port group, Management Network, Management Traffic,etc….., put the explanation between parenthesis.
Vmware is confusing with its terminology.

Thanks
0
 
LVL 13

Expert Comment

by:Abhilash
ID: 39964878
FC Adapters are for storage traffic. Not network traffic. You will need NIC cards for network/Ethernet traffic.
Your terminology is correct. Management traffic will also involve the traffic that is used to access the host from its IP.
Your terminology is correct but why are you bringing fiber channel adapters. If you replace them with NIC cards in your comment you are absolutely right.

Or if i have to sum it up for you. You will need a NIC  each for Management traffic and vMotion.
You will need another pair for VM traffic.
Totally 4 of them for management, vMotion, and VM traffic.

Hope i made it clear for you.
0
 

Author Comment

by:jskfan
ID: 39964904
We have 4 HBAs , 2 goes to 2 separate FC Network Switches and 2 goes to 2 separate FC Storage Switches.
0
 
LVL 13

Expert Comment

by:Abhilash
ID: 39964939
Can you confirm if they are FCoE NICs?
You will not have just HBA's.
0
 
LVL 25

Assisted Solution

by:Mohammed Khawaja
Mohammed Khawaja earned 500 total points
ID: 39964974
VM network is used for networking and not storage.  Your FC cards are HBAs and used for storage which has nothing to do with VM network and is used by your Data Store.  Do not confuse networking with storage.  For VM network you need Ethernet adapters.  For iSCSI storage, you use Ethernet cards but it is not used for your VM Network usage.
0
 
LVL 62

Expert Comment

by:gheist
ID: 39965018
I wonder what is fiber channel network switch... Is it 10GbE ethernet or something less brainy?
0
 

Author Comment

by:jskfan
ID: 39965084
I am not at work…

But I am sure that the ESX host has 2 FC cables going to 2 separate Network switches and 2 other FC cables going to 2 separate switches for Storage.

The FC cables for Network and Storage look different though
0
 
LVL 13

Expert Comment

by:Abhilash
ID: 39965090
Yes they should look different. One pair would be either 1Gig or 10Gig Ethernet cables :)
And other pair would be FC uplinks.
0
 

Author Comment

by:jskfan
ID: 39965100
fcfc
0
 

Author Comment

by:jskfan
ID: 39965103
The black cables for Storage I believe and the orange ones for Network

but they are plugged to 4 HBAs….2 HBAs go to 2 separate Switches for Network
2 other HBAs go to 2 separate switches for storage
0
 
LVL 62

Expert Comment

by:gheist
ID: 39965134
Exactly other way around. Black one has SFP module on both ends.

So you have 10GbE
Set it in one of failover modes and use netioc to make sure two vmkernel services and user network dont clash.
0
 

Author Comment

by:jskfan
ID: 39965463
what do you mean by vmkernel services and user network…..these are other terms of technology….(:)
0
 
LVL 25

Expert Comment

by:Mohammed Khawaja
ID: 39965503
Both pictures are fiber cables used for connecting to storage.
0
 

Author Comment

by:jskfan
ID: 39965566
In Fact these are the standard traffics that exist in Vmware:

**Management Traffic: Host to Host communication including VMHA and hosts to Vcenter communication.
Though in some Terminology Management traffic is for VMHA the other traffic is Management Network (Host to host to Vcenter)

Well do I need one Vmkernel port to Management Network and one Vmkernel to Management traffic ?

**Vmotion Traffic
do I need separate Vmkernel for Vmotion?

**IP Storage Traffic
Do I need separate Vmkernel for Storage Traffic?

These are the most common traffic (I did not cite Fault tolerance Traffic)

How many NICs should I have for the traffic separation and redundancy, considering that I already have 4 HBAs in each ESX host, 2 connected to 2 separate Network Switches and 2 to 2 Storage Switches
0
 
LVL 124

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 39966470
We double up ALL network services for ethernet and fibre channel.

so we have at least 2 nics per Management Network, vMotion, iSCSI, Virtual Machine Network, these could be actual physical network connections, or two nics, and many VLANs, if using 10GBe.

if using 1GBe, it's many nics for resilience, with no VLANs.

e.g.

nics!
You will need a pair for vMotion (VMKernel) and a pair for iSCSI Storage Traffic, but do you use iSCSI ?

if you are using FC, you do not need a pair for iSCSI Storage Traffic, because your storage traffic goes via FC.

But vMotion goes through ethernet.
0
 

Author Comment

by:jskfan
ID: 39966622
it looks Good…I do not see where VMHA goes
0
 

Author Comment

by:jskfan
ID: 39966632
I guess VMHA can go with VM Network
0
 
LVL 124
ID: 39966671
Management Network!

HA Traffic (e.g. communications and heart beat)
0
 

Author Closing Comment

by:jskfan
ID: 39967192
Thank you Guys
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

HOW TO: Connect to the VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere (HTML5 Web) Host Client 6.5, and perform a simple configuration task of adding a new VMFS 6 datastore.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to install log collectors and how to configure ESXi 5.5 for remote logging Open console session and mount vCenter Server installer: Install vSphere Core Dump Collector: Install vSphere Syslog Collector: Open vSphere Client: Config…

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question