Solved

Multipathing in VMware kills connection to datastores

Posted on 2014-09-26
20
477 Views
Last Modified: 2014-10-02
Hello Experts - I recently upgraded to ESX/Vcenter 5.1 and I'm trying to add a second NIC to my iSCSI vswitch in VMware.  Whenever I enable the second NIC within five minutes I lose connectivity to three of my six datastores.  My environment consists of three ESX 5.1 hosts, Vcenter 5.1, a Netapp FAS2240-4 SAN, and an HP 2920 switch.  After speaking with support engineers from both companies I believe the problem relates to load balancing.  I have not been able to determine the answer to certain questions:

1) Which SATP software should VMware be using to connect to the Netapp?  Right now its showing up as VMW_SATP_DEFAULT_AA which I am told is generic software when VMware doesn't know what kind of SAN is on the other end.

2) Is the FAS2240-4 able to use ALUA (I don't believe it is capable)

3) What Load Balancing method should I be using and where should it be set?

For number three I have been told different things by different techs  Initially we configured the VMware datastores to use round robin and also set the ifgroups on the Netapp to use round robin but I was told that this should only be set on the VMware side, not the Netapp.  The Netapp tech that assisted in the initial setup seemed to think both the Netapp and Datastores needed to be setup to use round robin so that is how we did it.  As much as I'd like to just try disabling load balancing on the Netapp it appears that the ifgroups can't be modified once they are created...true?

I've got so much conflicting info here that I'd like to try and get a consensus on what the actual best practices are for my particular configuration.  I'd really appreciate any advice on how to get this going.  I've been through 4 VMware techs so far without any resolution.
0
Comment
Question by:danbrown_
  • 10
  • 9
20 Comments
 
LVL 118
ID: 40346043
Round Robin, with single IP Address on the NetApp iSCSI VIF

and multi path setup as per my EE Article

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0

and Jumbo Frames

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client

that how we have our files configured.

Also install Virtual Storage Console because it will configure multipathing and iSCSI settings for you to NetApp preferred values.
0
 
LVL 28

Expert Comment

by:asavener
ID: 40346044
Personally, I would not use load balancing.  I would have multipathing for failover, but I'd turn off round-robin.

One of our techs went to VMworld a few years ago and came back with that recommendation.
0
 

Author Comment

by:danbrown_
ID: 40346051
The VMware tech recommended using the Fixed Path Selection but all of my ifgroups on the Netapp are set to use round robin.  How can I change this on the Netapp without recreating the ifgroups?
0
 
LVL 118
ID: 40346066
You will need to re-create the ifgroups and IP Address.

e.g. rdfile /etc/rc

and wdfile /etc/rc

or use Oncommand to edit the network interfaces.
0
 

Author Comment

by:danbrown_
ID: 40346074
@Andrew - I went through your article and that is exactly how we had things configured in VMware and on the Netapp.  The problem is when doing that I lose connectivity to the datastores.
0
 
LVL 118
ID: 40346096
do you have a single IP specified for iSCSI connections ?

do you have singe IP Address on the NetApp VIF (igrp)

did you apply the recommended values for iSCSI, or via Storage Console for MPIO ?
0
 

Author Comment

by:danbrown_
ID: 40346183
do you have a single IP specified for iSCSI connections ?
Here is how my vSwitch looks in VMware (one NIC is disable due to the problem:
vswitch

 do you have singe IP Address on the NetApp VIF (igrp)
Yes, here is a screenshot of the config
ifgroup 1-2ifgroup 3-4
 did you apply the recommended values for iSCSI, or via Storage Console for MPIO ?
You recommended using Round Robin, VMware says to use Fixed.  Using Round Robin I'm losing connectivity to the datastores.
0
 
LVL 118
ID: 40346202
You have TWO IP Addresses specified for iSCSI on the filer!

We do not do this, we use one for iSCSI (and two for NFS)

and we ensure that the IP Address is trunked across all four ports on the Filer via LACP on the physical switches.

also make sure that your physical network ports, are standard ports, not trunked, LACP etc

We chose to apply configuration via Storage Console, because it alert if not correct, apply, and then reboot server.
0
 

Author Comment

by:danbrown_
ID: 40346223
OK, let me restate this so I am sure I understand your recommendation.  I should remove the existing ifgroups from both Netapp heads.  I should then create just one ifgroup containing all four NICs, one for the top controller and one for the bottom controller.

I don't understand your next recommendation.  First you say use LACP for the filer ports on the physical switch (HP 2920).  The next line says make sure the ports are standard and not trunked or using LACP...which is correct?
0
 
LVL 118
ID: 40346246
Filer

Trunk Four Physical Ports (if you have LACP use it!)

Hosts (ESXi)

Standard Access Ports 2 of, no trunk, no LACP

Setup as per my Article.

Work with a single Controller first,

Make sure Partner Addresses are specified for correct take over and give back of both controllers.
0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 

Author Comment

by:danbrown_
ID: 40355179
It turned out most of this was not necessary.  Here is a screenshot of how my vSwitches are configured now:

vswitches
And the Netapp:
netapp.jpg
0
 

Author Comment

by:danbrown_
ID: 40355353
I've requested that this question be closed as follows:

Accepted answer: 0 points for danbrown_'s comment #a40355179

for the following reason:

Found own solution
0
 
LVL 118

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 40355340
That's not best Practice. You have no fail over. If you look at all the VMware documentation, My EE Article, and NetApp documentation, it's not supported, and not best practice.
0
 

Author Comment

by:danbrown_
ID: 40355348
Except my setup does not work when configured using best practices, this at least works.
0
 
LVL 118
ID: 40355354
Your configuration must be incorrect, you only need a single IP Address for controller. (not two!)

and I'm actually going to object, because you have trunked ALL four nics on the NetApp as per my post.

Have you had your configuration of the NetApp and VMware "runner stamped" by Professional Services, e.g. NetApp Vendor Engineers ?

or did you create it yourself ?
0
 

Author Comment

by:danbrown_
ID: 40358115
I actually left the VIFs as they were but changed the IP address on two of them (one top and one bottom).  However you've always been a real help in the past so if you want the points they're all yours!  Thanks for all of your help, very much appreciated.
0
 

Author Closing Comment

by:danbrown_
ID: 40358118
My setup did not work in this configuration but Andrew has described the accepted best practice in detail which is how the initial setup should be configured.
0
 
LVL 118
ID: 40358147
Dan

Unless you have a reason to use iSCSI, NetApp recommends NFS and favours NFS these days, NFS has less overhead and performs better now than iSCSI!

NetApp Filers were originally built around NFS, and now we have the VAAI NFS Plugin from NetApp we can do Thick on NFS, and datamoves based on snapshots, so clone/copies are faster and handled by the NetApp Filer, and not the ESXi server!

It's worth considering, as you do not have to worry about LUNs, LUN Reserves, Snapshot Reserves etc

for VMware vSphere

We are moving Customers from iSCSI to NFS on NetApp!
0
 

Author Comment

by:danbrown_
ID: 40358153
Tempting, but after all the time it took to get setup using iSCSI (which the Netapp engineer who assisted with the setup actually recommended) I'm going to stick with it for now.  Maybe a slow transition over to NFS as things get migrated since it supports both.  Thanks again Andrew!
0
 
LVL 118
ID: 40358167
If you've got licenses for both and time try it!

I'm always here to help!!!

All the best

Andy
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

#Citrix #XenApp #Citrix Scout #Citrix Insight Services #Microsoft VMMAP #Microsoft ADEXPLORE #Microsoft RAMMAP #Microsoft TCPVIEW #Microsoft AUTORUNS #Microsoft PROCESS EXPLORER #Microsoft PROCESS MONITOR
This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
Teach the user how to install log collectors and how to configure ESXi 5.5 for remote logging Open console session and mount vCenter Server installer: Install vSphere Core Dump Collector: Install vSphere Syslog Collector: Open vSphere Client: Config…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

24 Experts available now in Live!

Get 1:1 Help Now