Jumbo Frames on NIC in VMware

Do I need to enable Jumbo Frames on the dedicated iSCSI NICs in VMware?

I have configured the MTU on the Dell SCv2020 SAN, Dell N3024 Switches, vSwitch1 and 2, but when I try to configure the NIC with MTU of 9000... This is the error I receive.

2015-10-30_13-42-14.png
If I flip the MTU on the NIC dedicated to the iSCSI connections back to 1500 then I can see all the LUNs and Datastores.  I have reviewed this document too.

http://www.experts-exchange.com/articles/9132/HOW-TO-Enable-Jumbo-Frames-on-a-VMware-vSphere-Hypervisor-ESXi-5-0-host-server-using-the-VMware-vSphere-Client.html

The NICs for this Host are:  Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet.

Thanks!
LVL 3
Wes FieldsAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I would always recommend the use of Jumbo Frames, on a VMKernel Portgroup and physical network interface, which is delivery iSCSI via a software adaptor.

Jumbo Frames need to be enabled everywhere in the chain, nics, server, san.

Also you may want to test with and without, and see which works best!

I have an article here which describes how do do that...

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0

These EE Articles are also applicable to 6.0.

Now, from your post, you mention you are using BCM5720 - these network interfaces may not support Jumbo Frames, or you have a mismatch of jumbo frames on your network. e.g. switch, SAN etc

You are using the software iSCSI Adaptor ?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Just an update, I've checked all our Dell Servers, running ESXi 6.0, and they are using BCM5720 wth iSCSI software adaptors, enabled for Jumbo Frames with an MNT of 9000, with no issues.

Check your SAN and Network settings.

Make sure ALL the VMKernel Portgroups are 9000, and also the main vSwitch is 9000.
Wes FieldsAuthor Commented:
Yes I am using the software iSCSI Adapter.  I have confirmed everything is running at 9000 MTU.

SAN
SWITCH
vSwitch

but the moment I switch the MTU on the vmNIC dedicated to the ISCSI it can't see any LUNs/datastores.  I checked the BCM5720 and it does support Jumbo Frames.
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Wes FieldsAuthor Commented:
Let me check one more time since your update Andrew.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
vary the size of the MTU, 4000, 8192, 9016, etc

a jumbo frame is defined as any frame larger than 1500 (approx!).

there is not really any standard.

you can also test your network using vmkping.

see here

https://communities.vmware.com/message/2418768

http://rickardnobel.se/troubleshoot-jumbo-frames-with-vmkping/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003728
Wes FieldsAuthor Commented:
Wes FieldsAuthor Commented:
I will start running through those documents Andrew and see what I can find.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
make sure the VMKernel iSCSI Fault Domain is also 9000, and not defaulting and set to 1500!

also most Dell storage switches we've seen 9000 are the default, but just check the individual ports!
Wes FieldsAuthor Commented:
I was able to resolve this issue tonight with the help of Dell Enterprise Technical Support.  The MTU size on the Dell N3024 must be set to 9216.

After I made the change to the switches and confirmed everything else was set to 9000 MTU, I could see all the LUNs on the 2 Host.

Console into Switch
Enable
Show System MTU
System Jumbo MTU 9216
copy running-config startup-config

Thanks for all the help with the links.  I was able to vmkping from the host successfully with vmkping -s 8972 -d x.x.x.x
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
that's odd we use 9000 on out ESXi nics, and the switches are set to 9216!

and we also use Dell switches!
ICT_ProCommented:
I belive switches MTU need to set a bit higher compare to client for overflow.
Wes FieldsAuthor Commented:
This was the fix after working with Dell Enterprise.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.