We help IT Professionals succeed at work.

Who or What is Sucking Down Bandwidth?

1,292 Views
Last Modified: 2011-10-19
Full T1 pipe from LAN to Internet. At off hours, when download usage is checked by ISP, less than 1% is generally where it stays. When I run a bandwidth test from a test website, it is consistently around 1400 kbps down and up. During business hours, usage check by ISP can show in the 85% range. Test websites show numbers like 400kbps down and 800kbps up (these dont provide accurate gateway speeds because they involve LAN bandwidth). I notice the latency when I login remotely. I would like to find an app that can monitor the LAN and tell me what nodes are using how much Internet bandwidth. Then I can track down who/what is the bandwidth hog. There is an Exchange Server and a Terminal Server operating on the LAN, the only authorized apps/machines using Internet bandwidth. The TS users, no more than 10-12 at a time, connect using the Windows 2000 TSAC which uses little bandwidth.
Comment
Watch Question

Lee W, MVPTechnology and Business Process Advisor
CERTIFIED EXPERT
Most Valuable Expert 2013

Commented:
You do realize that Terminal Services uses 30 Kbps per connection, give or take... so 10-12 is 300-360 Kbps... plus printing can saturate the line.

Now, you can install the Network monitor application on the Terminal Server and get a near exact picture of how much bandwidth it is using.  If it's not your primarily culprit, then I'd also suggest checking your managed switch - you do have a managed switch and not some cheap "dumb" switch, right?  If you check the managed switch you should be able to get an idea of what ports are the "busiest".  Failing that, you can explore a tool like MRTG and enable SNMP on the workstations so you graph what system is using what kind of network bandwidth.

The problem is most network analysis software that can EASILY do what you want and present it in a nice interface will cost you significant money.  The cheap methods, which I assume you're looking for, are going to be labor intensive.
Kamran ArshadIT Associate

Commented:
Hi,

For Network Monitoring you may use any of the below applications:

HP-OpenView      www.hp.com      Propriety
Microsoft Operation Manager      www.microsoft.com      Propriety
Cacti      www.cacti.net      RRDTool
MRTG      oss.oetiker.ch/mrtg      RRDTool
PRTG      www.paessler.com/prtg      NetFlow/RRDTool
Adventnet OP Manager      www.adventnet.com      Propriety
Adventnet Netflow Analyzer      www.adventnet.com      Using Cisco NetFlow
SolarWinds Orion      www.solarwinds.com      Propriety
BigBrother      www.bb4.com      For Linux/Unix based machines
CiscoWorks      www.cisco.com      Best for Cisco devices
Observer      www.networkinstruments.com      Propriety
AutoNOC      www.autonoc.com      Propriety
ServerAlive      www.woodstone.nu      Propriety
SNMPc      http://www.castlerock.com/      Propriety
Traffic Sentinel      www.inmon.com      Sflow
WhatsupGold      http://www.whatsupgold.com/      Propriety
AirMagnet      http://www.airmagnet.com/products/laptop_analyzer/      Propriety
CommView      http://www.tamos.com/products/commview/      Propriety
SolarWinds NetFlow Analyzer      www.solarwinds.com      Netflow
Scrutinizer NetFlow/Sflow Analyzer      www.plixer.com      NetFlow/Sflow
NetXMS      http://www.netxms.org/      Propriety
IBM Tivoli        www.ibm.com      Propriety
Open-Source            
NAME      URL      TYPE
Ntop      www.ntop.org      LAMP based NMS with Windows port available
Bandwidthd      bandwidthd.sourceforge.net      LAMP based NMS
ZenOSS      www.zenoss.com      LAMP based NMS
Nagios      www.nagios.org      LAMP based NMS
JFFNMS      www.jffnms.org      LAMP based NMS
OpenNMS      www.opennms.org      LAMP based NMS
Zabbix      www.zabbix.com      LAMP based NMS
BigSister      www.bigsister.ch      For Linux/Unix based machines
Etherape      etherape.sourceforge.net      LAMP based NMS
GroundWork      www.groundworkopensource.com/      LAMP based NMS
NAV      metanav.uninett.no      LAMP based NMS
Netdisco      netdisco.org      LAMP based NMS
ODCNMS      www.odcnms.org      LAMP based NMS

For Bandwidth Monitoring, you can use the below applications:

IPerf                        dast.nlanr.net/Projects/Iperf/
QCheck                   www.netiq.com/Qcheck/default.asp
I need the Windows 2000 cd to install the files needed for Network Monitor. I do not have it.  Several of the ports on the 3Com SuperStack 3 3300 XM switch run to other downstream switches which have their own PCs and devices attached to them, so that wouldnt be too conclusive.
As far as TS users is concerned, right now there are twelve users connected and that still leaves me with >1000k/sec. So I dont think that is the problem. Anybody? Hello?
This open question is from July 2008, not 2009. I just received an alert regarding it, but is it still a relevant question? Possibly Vee_Mod did not notice the year.
In that case,

Based on the information presented so far, what I would do is grab the freeware edition of PRTG from www.paessler.com, which allows for monitoring of 10 nodes. Have it monitor the downstream ports on the one managed switch in an effort to determine what section that the bandwidth is going to.

It will not point out immediately what machine or server is suspect, but at least it will point you in the direction where to start. From there, if a machine is suspect, you could always patch it up directly to the managed switch to confirm.
Is Net Flow required for what you describe? Cisco 2960s do not support it.
No. While PRTG supports Netflow, it is a separate license.

PRTG supports standard SNMP traffic libraries out of the box. Simply create a read SNMP community on your Cisco using

snmp-server community communityname RO

Then inside PRTG
1. Choose Add Device
2. Pick a group to add the device to
3. In the next screen, name the device and supply the IP of an interface on the Cisco
4. Uncheck the box next to "Inherit Credentials for SNMP Devices" and supply the community name you used in the Cisco

Now the device is created, you need to tell PRTG to monitor specific ports. Choose the device, then click Add Sensor
- Choose the radio button for SNMP traffic
- PRTG will query the Cisco and present to you a list of interfaces
- Select the ports you want to monitor. Watch out here, as with Cisco, the VLANs are listed first, then the physical ports below. If you have desc fields set on your interfaces in the Cisco, SNMP will show those.
- Watch out with the difference between In and Out. The default In and Out is correct when you are thinking in terms of your T1, but for your internal ports, you need to reverse the description if you want to view the traffic as it relates to the T1. This is because traffic leaving the switch going to the T1 is indeed going out, but traffic leaving the switch going to a computer is actually coming in from the T1.

Hope that helps.
Adrae
CERTIFIED EXPERT
Top Expert 2014

Commented:
The 2960's may not support netflow, but they do support SPAN.  You could mirror the port that your Internet router is connected to, then install NTOP on a linux box and connect it to the port that is the traffic is mirrored two.  ntop can then produce reports that can tell you who (IP address) is doing what.
The problem that spawned this question was what I suspected to be someone/something on the LAN using most of the bandwidth to the point of considerable packet loss. As time went on, I opened a ticket with the ISP that was open for almost 2 months. They threw everything they had at it culminating in building a new circuit side by side the old one and switching us to the new one in 12/08. That seemed to solve the problem, but they never could say what the problem was. Didnt think about it again until I started to notice the same symptoms a few weeks ago and high download utilization during business hours.

Just got all new Cisco hardware last month, and I was thinking switches because I started a trial of OpManager because they told me I could use the Switch Port Mapper to nail down what port(s) were using the most b/w without using NetFlow. But that was a waste of time because the guy I that told me that was wrong, NetFlow is needed for that. Then, DUH!, I realized I should be using the ASA 5505 to track the problem. Spent all weekend trying to d/l and run the asdm demo so I could see what it does, without any luck. Spoke with Cisco TAC today, they were having problems with the site over the weekend. Couldnt d/l asdm-demo-6.21.msi but if I installed the 621 bin file it would go as far as telling me the top 10 b/w users. Or I could upgrade the IOS from 8.0(3) to 8.2(1) which would include NetFlow. The upgrade procedure is rather involved and for someone like me who's Cisco skills are out of shape, a little scary, so Im not sure if its worth it.

I could shoot the wad and use the PRTG trial which is only good for 30 days but would give me 500 nodes. I take it I could monitor all in use ports on all the 2960s with that?
Using SPAN with NTOP sounds like it could be an option, is Linux a must for that?
Anyway, thanks to Vee Mod for reopening the question. That wasnt necessary, but between the new Cisco h/w and these 2 responses, a resolution to the original question is close at hand. Not sure which of these options is the best way to go.
Yes, the trial with 500 nodes will give you the ability to watch all the switch ports. One port = one node in PRTG. We use the 500 ourselves in our data center to watch all the tier 1 and tier 2 switches.

It will be an easy setup too, since you can just select all the ports and add them, you're not concerned about renaming them or making reports, just raw dumps, so I'd figure maybe half an hour of effort plus adding the snmp communities to your equipment and you'd have your answer in a hurry.
I started using the ASDM for the ASA 5505. But the closest I can get to identifying suspects so far is looking at a usage pie chart which gives an outside IP then I have to scan the syslogs in realtime and try to find that IP and a corrosponding internal IP before the window passes. Not very scientific. So it sounds like PRTG is next. Did you mean the Network Monitor or the Traffic Grapher?
CERTIFIED EXPERT
Top Expert 2014

Commented:
You can get NTOP for Windows or just download the source and compile your self.  Here is a link for HOWTO:

http://www.ehow.com/how_4468882_install-ntop-windows.html
I was having trouble trying to do what it is I need to do so I decided to RTFM. A reality check please.
 It sure looks like I cant use PRTG to track down who/what is using all my bandwidth with SNMP. I would love to be proven wrong.

PRTG-bandwidth-monitoring.gif
I have 200 active sensors deployed.
CERTIFIED EXPERT
Top Expert 2014

Commented:
Its not PRTG's issue.  Its SNMP's, devices don't monitor traffic by IP address and port, unless they have NetFlow enabled, but NetFlow does not have SNNP interface.

So it does not matter what you get; PRTG, MRTG, NAGIOS, Solar-Winds, OpManager, ect., if they rely on SNMP they can't tell you who is using what bandwidth.

Well, sort of they could, if you monitor each and server network interface (switch port, sub-interface, NIC on a computer) you can see what each ports in and out byte count is and if you divide the total # of bytes over query interval you have % utilization.  But you just don't know where the traffic is going to or coming from.
To me the idea is.. you have a PC, or a printer, or a file server, or what have you.. is connected to a switch port. Switch ports don't share. So then PRTG pulls traffic data from those ports, and you are able to see which ports are pulling high amounts of traffic.

Once you figure that part out, THEN you go the speific port/machine and you start using tricks like wireshark, or simply asking the user WTF they are running on their computer. Wireshark the port as a mirror and you'll see all the traffic doing through it.

But PRTG + the ports will help you narrow down the WHERE and then you can localize with Wireshark and arp and local tools, nmap, to find the WHAT. You can combine these both into one with Netflow but I find that harder.
I guess what I'm trying to say in my last post is look at your PRTG sensors and see where the large chunks of traffic are moving from. Then go to those specific machines or switch and start netflowing or wiresharking there, as you'll have a much smaller base to which to try to capture traffic from. Then you can determine what the traffic is, is it malicious in, or stupid users out, and firewall it.
SNMP is insufficient for what I need, I dont want to be shooting in the dark, dont have the time for that. Ive upgraded the IOS on my ASA 5505 to 8.2 which supports NetFlow. Thats the way to go unless anyone sees a problem with that.
Forgot to add sorry for the delay on a response, lots of fires to put out.
Istvan KalmarHead of IT Security Division
CERTIFIED EXPERT
Top Expert 2010

Commented:
what type switches do you have?
see my comment 7/10, 6:16

Cisco 2960's
I opened this question over a year ago. Ive tried many things. Recently got new Cisco firewalls and switches, I thought that would breathe new life into a possible solution. After upgrading to 8.21 on the ASA, Cisco told me that could still not accomplish what I want. All that got me was a recommendation from Cisco that I downgrade now because 8.21 has too many bugs.

That about does it for me. I dont usually give up but its sucked as much out of me as Im gonna let it. Unless someone has an answer that is top secret, Im gonna close the question.

(I wonder if a question open this long is a record for EE)
CERTIFIED EXPERT
Top Expert 2014

Commented:
In order to see who (by IP address and TCP/UDP port) is doing what you either need NetFlow or to mirror the port that your firewall is on and forward the traffic to NTOP (or a NTOP type product).



I don't have ASA 8.2 so I am not sure if there are restrictions on what it can do in regards to NetFlow.  I did find one site that described how to set it up:

http://www.cisco-tips.com/cisco-asa-netflow-support-%E2%80%93-netflow-security-event-logging-nsel/
Yeah, but when a CCIE certified Cisco TAC tech on the firewall team tells me it wont work, thats all I need to hear.

But as a final attempt,  Im willing to try your suggestion about NTOP. Got a how-to link?


CERTIFIED EXPERT
Top Expert 2014
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
Im  still trying to find the time to set up the Linux box. Any particular flavor of Linux?
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Commented:
I think the Linux Idea is still the more long term and economical solution though if you are on a tight budget.
CERTIFIED EXPERT
Top Expert 2014

Commented:
I personally use Fedora, but any of them will do.  I would suggest using one that supports installing software using RPM (Redhat Package Manager) or apt.  I am fairly sure that most of the popular distributions will support one, if not both, of these.  

RPM and apt are used to manage (install, update, remove) software packages/programs.  It makes life a lot easier if your are not really into building (compiling) programs from the source.

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.