We help IT Professionals succeed at work.

How to Enable flow control on VMware ESXi 4.1 and Persist the config across reboots

Hi All,

I intend to enable flow control (along with Jumbo frames) between my VMware ESXi hosts and my Cisco 3750G switches in a multipathing iSCSI setup.

VMware version is 4.1, ESX Hosts are HP Proliant DL 380 G7 servers with Broadcom NICs

Questions:
------------------
1, How do I configure flowcontrol transmit on (send on) and receive off using the ESXi CLI
2, How do i save the above configuration to persist across ESXi host reboots?

All the above instructions should be provided step by step.

3, If I am using etherchannel between Cisco 3750G switches and my storage appliance (NetApp FAS2020) is it advisable to confiure etherchannels also between my VMware ESXi Hosts and the Cisco Switches? If yes, which type of etherchannel (mode on or mode active ) should be configured?

VMware vSphere 4.1 does not support lacp etherchannels (I am told) which is what I have btn NetApp and Cisco.

Each ESXi hosts has two NiCs for iSCSI traffic connected into separate Cisco switches in a cross-stack.



Simple straight forward asnswers would be most appreciated.

Regards.
Comment
Watch Question

Andrew Hancock (VMware vExpert / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
see my EE Article for Best Practice with Step by Step and Screenshots

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 4.1

Enabling Jumbo Frames is quite straight forward at the console

login into the server via ssh

try esxcfg-vswitch -l


~ # esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         64          18          64                1500    vmnic0,vmnic1

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Management Network    0        1           vmnic0,vmnic1
  VM Network            0        14          vmnic0,vmnic1

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch1         128         1           128               1500

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VLAN 2                2        0
  VLAN 5                5        0

this will display the above output

then to change to jumbo frames set mtu to 9000

esxcfg-vswitch -m 9000 vSwitch1

and then check with esxcfg-vswitch -l again

 esxcfg-vswitch -l
Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch0         64          18          64                1500    vmnic0,vmnic1

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  Management Network    0        1           vmnic0,vmnic1
  VM Network            0        14          vmnic0,vmnic1

Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch1         128         1           128               9000

  PortGroup Name        VLAN ID  Used Ports  Uplinks
  VLAN 2                2        0
  VLAN 5                5        0


I would only change the mtu on a virtual swtich and networking that's configured for jumbo frames, unless you want to change the default vSwitch0.

Yes, I would trunk and use Etherchannel

http://kb.vmware.com/kb/1004048

Author

Commented:
Thanks  Hanccocka.

I am familiar with your article on setting up the software iSCSI Adapter for Multipathing.

I believe the pointer to the ESXi Etherchannel KB is also useful.

Can you comment on Question 1 and 2 also

 How do I configure flowcontrol transmit on (send on) and receive off using the ESXi CLI
2, How do i save the above configuration to persist across ESXi host reboots?
VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017
Commented:

Author

Commented:
Hi Hancoccka,

I have taken a look at KB1013413 earlier but my difficulty was how to locate the default datastore containing the configuration files.  I have my hosts connected to SAN storage now and all I see are the datastores on the SAN. Is there a default path that I can use on the ESXi CLI to access the default datastores containing configuration files on the ESXi Hosts?

Secondly, VMware KB1004048 suggests that the load balancing for the vSwitch in ESXi link aggregation should be IP Hash based and nic teaming should be configured with two active adapters (assuming two nics for multipathing). This is different from the NIC teaming Tab configuration setup used in your article on setting up the software iSCSI Adapter for Multipathing.

Should I use the Nic Teaming tab configurations suggested by  VMware KB1004048  because I am sething up ESXi link aggregation with Cisco static etherchannel?
Andrew Hancock (VMware vExpert / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017
Commented:
Flow control configuration uses the esxcfg-module  command?

what configuration files are you looking for, do not get confused with ESX, you have ESXi.

I would setup as per Etherchannel and IP HASH.

Author

Commented:
Hancoccka,

I have broadcom NICs so I suppose the right tool was ethtool and not esxcfg-module.

if you agree about ethtool, KB: 1013413 mentions the below hence my question about where to locate the /etc/rc.local and other files

" 5.First make the /etc/rc.d/rc.local file executable by setting the permission with command "chmod 755 rc.local " and then open that /etc/rc.d/rc.local file using a text editor. For more information, see Editing configuration files in VMware ESX (1017022).

Note: You can verify if the rc.local file is correct by verifying /etc/rc/d/rc3.d file and check the directory listing with ls -l under /etc/rc/d/ directory and verify which file s99local is pointing to.


6.Append the same commands used in Step 4, placing each on its own line.


7.Save the file.


8.For an ESXi host, save the configuration change using the command:

# /sbin/auto-backup.sh
"
Andrew Hancock (VMware vExpert / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
those config changes are for ESX.

Author

Commented:
thanks for the clarification Hancoccka.

Before I close this question however, can you further clarify if after using the esxcfg-module command for flowcontrol setup, there is noting else to do to make the configs permanent?
Andrew Hancock (VMware vExpert / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017
Commented:
esxcfg-module will make the changes permanent.

Author

Commented:
Hancoccka,

What is your opinion on the Broadcom Adapter section at the link below for flowcontrol configs?

http://www.sysadmintutorials.com/tutorials/vmware-vsphere-4/esx-4/enabling-flow-control-with-vmware-esx-4/
Andrew Hancock (VMware vExpert / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
with any storage configuration you need to test, what works for your setup.

Explore More ContentExplore courses, solutions, and other research materials related to this topic.