Jumbo Frame Support

A server admin requested that I enable Jumbo Frame Support on a Cisco 3750 switch that is used for an Oracle interconnect network between two servers that are located on two HP Blade enclosures.
I entered the command "system mtu jumbo 9000"
I then rebooted the switch to take effect.
When I ran "sh system mtu" the system MTU is 9000 but the routing MTU and interface MTU is still at 1500.
What is the difference between the system MTU and the routing and interface MTU's?
Did I accomplish what is needed for the Oracle interconnect network between the servers?
Dragon0x40Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dragon0x40Author Commented:
Found some information: http://www.oracle.com/technology/products/database/clustering/pdf/EMC_BestPracticeforOracle.pdf


Jumbo frames
Maximum Transfer Unit (MTU) sizes of greater than 1,500 bytes are referred to as jumbo frames. Jumbo
frames require Gigabit Ethernet across the entire network infrastructure—server, switches, and database
servers. Whenever possible, EMC recommends the use of jumbo frames on all legs of the storage or RAC
interconnect networks. For Oracle RAC 10g installations, jumbo frames are recommended for the private
RAC interconnect to boost the throughput as well as to possibly lower the CPU utilization due to the
software overhead of the bonding devices. Jumbo frames increase the device MTU size to a larger value
(typically 9,000 bytes). Celerra Data Movers support MTU sizes of up to 9,000 bytes.
Typical Oracle database environments transfer data in 8 KB and 32 KB block sizes, which require multiple
1,500 frames per database I/O, while using an MTU size of 1,500. Using jumbo frames reduces the number
of frames needed for every large I/O request and thus reduces the host CPU needed to generate a large
number of interrupts for each application I/O. The benefit of jumbo frames is primarily a complex function
of the workload I/O sizes, network utilization, and Celerra Data Mover CPU utilization, so it is not easy to
predict.
Detailed instructions on configuring jumbo frames is contained in the Oracle Database 10g/Oracle RAC
10g Celerra NS Series NFS Applied Technology. For information on using jumbo frames with the RAC
Interconnect, please see MetaLink Note 300388.1.
0
harbor235Commented:
Depends on the interface, all 10/100 interface support up to 1998 on the interface, GigE interfaces support up to  9000 bytes. You cannot change the mtu on the interface you must do it via the system command;

for 10/100
system mtu 1998
exit
reload

for GigE
system mtu jumbo 9000
exit
relaod

Verify;
Switch# show system mtu
System MTU size is 1998 bytes
System Jumbo MTU size is 9000 byte

harbor235 ;}
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Rick_O_ShayCommented:
I don't know the specifics of this Cisco switch but generally the mtu size of routed interfaces on switches can not be set to jumbo nor can 10/100 ports.

If your server to server connection is layer 2 switched end to end you can use jumbo frames. If it crosses a routed interface it will be using the 1500 maximum.

Here is the Cisco doc on the subject. See the 3750 area for details and caveats.
http://www.cisco.com/en/US/products/hw/switches/ps700/products_configuration_example09186a008010edab.shtml
0
Get Cisco Certified in IT Security

There’s a high demand for IT security experts and network administrators who can safeguard the data that individuals, corporations, and governments rely on every day. Pursue your B.S. in Network Operations and Security and gain the credentials you need for this high-growth field.

harbor235Commented:

Correct, you cannot change the mtu vi interface configuration changes, however, you change it globally as stated in my last post and in the doc Rick posted above.

"Catalyst 3750/3560 Series switches support an MTU of 1998 bytes for all 10/100 interfaces. All Gigabit Ethernet interfaces support jumbo frames up to 9000 bytes. The default MTU and jumbo frame size is 1500 bytes. You cannot change the MTU on an individual interface. You must set the MTU globally. Reset the switch afterwards for the MTU change to take effect."

harbor235 ;}
0
Dragon0x40Author Commented:
thanks harbor235 and Rick O Shay,

I will read the link.

How often is this jumbo frame size used?

This is the first I have had to use it.

Any way to test it?
0
harbor235Commented:


sniff the traffic and decode the mtu

harbor235 ;}
0
Rick_O_ShayCommented:
I think jumbo frames are used mostly in server to server applications like you are seeing.
Your switch may have a port counter for jumbo frame stats.
0
mikecrCommented:
Jumbo frames are used to transmit large amounts of data across backbones or between servers. The only drawback is that all devices must be able to process jumbo frames. Sometimes however, they can be a waste. Once you set jumbo frames, client stations that are properly configured attempt to send everything as jumbo frames, including broadcasts. According to the client, it doesn't pick and choose what frame size to use on each transmission so it's not uncommon to have a 9000 byte frame with only 64 bytes of data in it.

I normally only recommend using jumbo frames for backup purposes or for replication of data between servers. You should refrain from configuring client workstations to use jumbo frames although they support it because this can cause strain on the switch that they are plugged into because each switch only has so much cache memory to support packet transmission and could essentially slow down client network transmissions.
0
Dragon0x40Author Commented:
thanks mikecr,

This is a switch in between two servers for an Oracle interconnect network.

I will pass along some more information when I receive it.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Switches / Hubs

From novice to tech pro — start learning today.