Cisco 6509 - usefulness of flow control on gig ports

Posted on 2013-06-21
Last Modified: 2013-06-24
Reading the best practices document on the 6509 it looks to me like flow control really only comes into play if a port buffer is over-loaded.  In practice is this actually like to happen if say the gig port maxes out at say 500Mbps of traffic??  l
Question by:amigan_99
  • 2
  • 2
LVL 20

Expert Comment

ID: 39266620
From the link:
The best use of this feature is on links between access ports and end hosts, where the host output buffer is potentially as large as the virtual memory. Switch-to-switch use has limited benefits.
I would agree with the statement. Switch to switch traffic shouldn't need to send pause frames ever unless you are exceeding the overall capacity of a switch, but when it comes to end devices flowcontrol has its benefits.

Author Comment

ID: 39266763
So say I have a Dell Server running ESX with 96GB ram, 50GB virtual memory and a gigabit over copper connection to a gig port on a 6509 switch.  The concern is somehow that the virtual memory would spew to the gig port on the 6509 at such a pace that a Pause might be needed so the 6509 port could empty and receive more data?  I am unclear why they mention virtual memory as being a potential source of too much data being sent in a time period.  It would seem like both sides would be governed at the NIC level - the host not able to send more than 1Gbps and switch not able to receive more than that.  Now if you had say 256 gigabit ports all receiving maximum traffic at the same time perhaps the backplane could not process everything - a blocking situation.  But the network I am talking about would be no where near this.  They probably have 10 servers and a single port might occasionally go to 300Mbps. I am trying to gather if Flow Control is really just a benefit in extreme circumstances - where a blocking architecture gets pushed to where blocking is actually happening.
LVL 57

Accepted Solution

giltjr earned 500 total points
ID: 39267946
One thing you have to remember when we talk about link utilization it is an avg. , not at absolute.  NIC's, along with CPU's, only have two speeds.  They have zero, which means nothing is going on and 100%, which means they are being used.

When you are sending any data over a 1 gigabit connection it is going at 1 Gbps.  Period, there is no other speed.  Your 300 Mbps comes from an avg. over time.  Say you avg. was 300 Mbps over a 5 minute period.  What this really means is that for about 1/3 of the time you were sending data t 1 Gbps and 2/3's of the time you were doing nothing.

Now, this does not mean for almost 2 minutes you were running at 100% link utilization, it could be 1 at 100%, 2 seconds at zero.  It could be 10 seconds at 100%, 20 seconds at zero.

Now for the virtual memory issue.  What Cisco is saying is that a OS could use a buffer that is equal to the size of the virtual memory on the system.  So in theory they believe that your ESX server could have a network buffer of 50GB and start sending 50GB of data at 1 Gbps.  

Now, in real life I don't know of a situation where the host's output buffer would be equal to the amount of virtual memory on a host.  There is so many things that virtual memory is used for that it could never be equal to.  I guess it could be close to it if you are not running a lot, but I would image that if you had enough data to send over the network to fill up the virtual memory, that you would have a program that was also using a lot of memory.

However, it could be bigger than the receiving device could handle in a short amount of time.  Depending on how busy the receiving device is.

In your case that may not ever happen, but you are looking at it from your situation.

But as you pointed out if you are exceeding the capacity of the switch because you have a very large environment you would need flow control.  

You may also have to take into account how quickly the receiving host is accepting data from the switch.  If the receiving host (or hosts) are super busy they may send a pause to the switch, which now has to buffer what it being sent to it.

Author Closing Comment

ID: 39270271
Thank you very much for the enlightenment on the topic.  

What I found for my buddy was that he had several iSCSI ports plugged right next to each other on a blade on the 6509.  And further I found that every eight ports had a throughput of 1Gbps - blocking architecture.  So I had him spread out the busiest connections and it sounds like things are much better.
LVL 57

Expert Comment

ID: 39270809
What you found is another issue.  With some switches and line cards a group of ports share resources, so you can't do line rate on every port.

On top of that iSCSI generally drives more traffic than "normal" Ethernet connections.

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Help with a subnetting question 7 58
Slow Internet Connection 9 57
Need network only 1 user? 10 72
VTP servers with 3650 switches 5 27
Don’t let your business fall victim to the coming apocalypse – use our Survival Guide for the Fax Apocalypse to identify the risks and signs of zombie fax activities at your business.
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Internet Business Fax to Email Made Easy - With  eFax Corporate (, you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

820 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question