Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Cisco 6509 - usefulness of flow control on gig ports

Posted on 2013-06-21
Medium Priority
Last Modified: 2013-06-24
Reading the best practices document on the 6509 it looks to me like flow control really only comes into play if a port buffer is over-loaded.  In practice is this actually like to happen if say the gig port maxes out at say 500Mbps of traffic??  l

Question by:amigan_99
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
LVL 20

Expert Comment

ID: 39266620
From the link:
The best use of this feature is on links between access ports and end hosts, where the host output buffer is potentially as large as the virtual memory. Switch-to-switch use has limited benefits.
I would agree with the statement. Switch to switch traffic shouldn't need to send pause frames ever unless you are exceeding the overall capacity of a switch, but when it comes to end devices flowcontrol has its benefits.

Author Comment

ID: 39266763
So say I have a Dell Server running ESX with 96GB ram, 50GB virtual memory and a gigabit over copper connection to a gig port on a 6509 switch.  The concern is somehow that the virtual memory would spew to the gig port on the 6509 at such a pace that a Pause might be needed so the 6509 port could empty and receive more data?  I am unclear why they mention virtual memory as being a potential source of too much data being sent in a time period.  It would seem like both sides would be governed at the NIC level - the host not able to send more than 1Gbps and switch not able to receive more than that.  Now if you had say 256 gigabit ports all receiving maximum traffic at the same time perhaps the backplane could not process everything - a blocking situation.  But the network I am talking about would be no where near this.  They probably have 10 servers and a single port might occasionally go to 300Mbps. I am trying to gather if Flow Control is really just a benefit in extreme circumstances - where a blocking architecture gets pushed to where blocking is actually happening.
LVL 57

Accepted Solution

giltjr earned 2000 total points
ID: 39267946
One thing you have to remember when we talk about link utilization it is an avg. , not at absolute.  NIC's, along with CPU's, only have two speeds.  They have zero, which means nothing is going on and 100%, which means they are being used.

When you are sending any data over a 1 gigabit connection it is going at 1 Gbps.  Period, there is no other speed.  Your 300 Mbps comes from an avg. over time.  Say you avg. was 300 Mbps over a 5 minute period.  What this really means is that for about 1/3 of the time you were sending data t 1 Gbps and 2/3's of the time you were doing nothing.

Now, this does not mean for almost 2 minutes you were running at 100% link utilization, it could be 1 at 100%, 2 seconds at zero.  It could be 10 seconds at 100%, 20 seconds at zero.

Now for the virtual memory issue.  What Cisco is saying is that a OS could use a buffer that is equal to the size of the virtual memory on the system.  So in theory they believe that your ESX server could have a network buffer of 50GB and start sending 50GB of data at 1 Gbps.  

Now, in real life I don't know of a situation where the host's output buffer would be equal to the amount of virtual memory on a host.  There is so many things that virtual memory is used for that it could never be equal to.  I guess it could be close to it if you are not running a lot, but I would image that if you had enough data to send over the network to fill up the virtual memory, that you would have a program that was also using a lot of memory.

However, it could be bigger than the receiving device could handle in a short amount of time.  Depending on how busy the receiving device is.

In your case that may not ever happen, but you are looking at it from your situation.

But as you pointed out if you are exceeding the capacity of the switch because you have a very large environment you would need flow control.  

You may also have to take into account how quickly the receiving host is accepting data from the switch.  If the receiving host (or hosts) are super busy they may send a pause to the switch, which now has to buffer what it being sent to it.

Author Closing Comment

ID: 39270271
Thank you very much for the enlightenment on the topic.  

What I found for my buddy was that he had several iSCSI ports plugged right next to each other on a blade on the 6509.  And further I found that every eight ports had a throughput of 1Gbps - blocking architecture.  So I had him spread out the busiest connections and it sounds like things are much better.
LVL 57

Expert Comment

ID: 39270809
What you found is another issue.  With some switches and line cards a group of ports share resources, so you can't do line rate on every port.

On top of that iSCSI generally drives more traffic than "normal" Ethernet connections.

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This program is used to assist in finding and resolving common problems with wireless connections.
In this article, we’ll look at how to deploy ProxySQL.
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…

610 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question