[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Using Ethereal to pinpoint a network throughput problem

Posted on 2005-04-11
26
Medium Priority
?
882 Views
Last Modified: 2013-12-07
Hi All

Getting sent out tomorrow to do some network analysis which I've only done a very little of.

Some rental place gave me an ancient version of sniffer pro 3.0.5 on a dinosaur laptop.  


The problem is that the client has a number of CCTV cameras mainly on 100 mb switches but there may be some 10s or 10 links.

When they up the cameras to 25 frames per second they get dropouts - in fact get dropouts at anything above 6.  They've got about 25 cameras and they said they've done 18 camera installs before on 100mbs networks - no worries.

So a few Qs: Pointing me to the right bits of the manual would be peachy
(oh and I have to operate in a strictly windows environment)


- Does ethereal handle switched networks and if so do I have to set anything special up.

- Can ethereal generate traffic and do throughput testing if so - how? and if not do people know of any other free tools I can use.  There used to be this thing called Raccoon speed tester that did timed transfers but I can't find it anymore on the web.  I'm guessing if it gets down to some switch not handling the type of packets I'll need something that can throw different packet types.

- Reporting - the client wants the problem pin-pointed and proved on paper.

- General tips and gotchas

- Should I send the sniffer pro back - is it going to give me anything important in this scenario that ethereal won't.

Sorry for all the Qs but I've been chucked in the deep end on this one and one of our techs has already hit this and bounced so it's up to me to save the day  :(


Much TIA
0
Comment
Question by:ausadmin
  • 9
  • 4
  • 4
  • +3
23 Comments
 
LVL 24

Accepted Solution

by:
purplepomegranite earned 500 total points
ID: 13757842
Ouch, I don't envy you!!

Ethereal will give you an insight into everything that passes the point in the network to which you connect.  It works by watching the network in promiscuous mode (i.e. captures all packets on the interface).  My guess is that it is only going to be useful in this situation if you find the right place to seat it.

It sounds to me that the network you are going to check has a bottleneck.  If it consists of many switches interconnected, the bottleneck is likely to be between these.  e.g. If you have a 100M switch with four devices hanging off it, but this switch is uplinked to another switch further up the line via one link, then each of the four devices connected to the first switch will only have an effective bandwidth of 25Mbps.  If switches are daisychained this is reduced dramatically.

I would start by getting an accurate network diagram - you will be able to work out any such bottle necks from this and may well not even need tools such as Ethereal.
0
 
LVL 28

Expert Comment

by:mikebernhardt
ID: 13757900
Many switches allow you to mirror traffic from one port to another, or from a group of ports to another. You'll need to do this to see any traffic on a switch.

I think the main thing you need to do is identify what percentage of bandwidth is camera and what is other things. Just because they've done 18 cameras elsewhere doesn't mean they can do 25 cameras here- it depends on the topology as purple said, and also on what else is using bandwidth. How much bandwidth is each camera generating? They should be able to tell you that. Or you can figure it out if you know image resolution and type of compression- there are tables out there that will give you bandwidth requirements.

25 cameras at 25 frames/sec could easily generate more than 100mb/sec traffic- and you can only expect a 100mb link to support about 60mb/sec without beginning to see a lot of congestion- indicated by dropped frames.
0
 
LVL 79

Assisted Solution

by:lrmoore
lrmoore earned 500 total points
ID: 13758164
I would bet that the primary issue you are going to find is duplex mismatches. You can easily find this by looking at port error counts on each switch port.
Problem in a nutshell:
Camera is only capable of 10M half-duplex, switchport set to 100/full-duplex = no connection
Camera is auto-negotiate 10/100 half/full, switchport set to 100-full = duplex mismatch
Camera is set to 10 half, switchport set to 10/full = duplex mismatch
Camera is set to 10/full, switchport set to auto = duplex mismatch
Camera is set to 100/full, switchport set to auto = duplex mismatch
Camera is set to 10/full, connected to 10Mb hub = duplex mismatch
Camera is set to either 10 or 100/full connected to 10/100 HUB = duplex mismatch
Camera is set to auto, switchport set to auto = OK
Camera is set to auto, connect to HUB = OK
Camera is set to 100/full, switchport set to 100/full = OK
Camera is set to 10/full, switchport set to 10/full = OK
Camera is set to 10/half, switchport set to 10/half = OK
Camera is set to 10/half, connect to hub = OK

Types of errors that you would look for are CRC errors, excessive collisions especially if the port is set to full-duplex..
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13758192
Heh, I'll bet £10 it's not duplex ;)  Most switches are autosensing, esp. 10/100 switches.
0
 
LVL 23

Assisted Solution

by:Tim Holman
Tim Holman earned 500 total points
ID: 13758204
Would agree with lrmoore - duplex issues are almost certainly the problem here, so you would need to log onto each switch and look at the port status.  The lights at the front may even help out - on a cisco they go orange for half-duplex and green for full duplex.  You want them all to be full duplex.
Anyway, to answer your questions:

- Does ethereal handle switched networks and if so do I have to set anything special up.

You need to setup a SPAN port on the switch, so that all traffic from all ports goes via the PC you've installed Ethereal on.

- Can ethereal generate traffic and do throughput testing if so - how? and if not do people know of any other free tools I can use.  There used to be this thing called Raccoon speed tester that did timed transfers but I can't find it anymore on the web.  I'm guessing if it gets down to some switch not handling the type of packets I'll need something that can throw different packet types.

Sort of.  An Ethereal capture will display usage summary, but it's only good for the time you capture, and to do throughput testing you really need to leave something running for a good 24 hours (at which point your Ethereal laptop will be full up and may keel over... worth a try though!).

- Reporting - the client wants the problem pin-pointed and proved on paper.

I suspect he'd want it fixed first!  You can document what you discovered, but that's about it.

- General tips and gotchas

You'll find these out as you go along... ;)

- Should I send the sniffer pro back - is it going to give me anything important in this scenario that ethereal won't.

No.  Ethereal is fine for point to point traffic capturing.  Sniffer Pro is better to use on Enterprise networks when you need to correlate data from several different network monitors.
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 13758313
>- Should I send the sniffer pro back
I wouldn't. The Expert analysis module is worth its weight in gold... Gives Symptoms/diagnoses
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13758314
Ok, my tenner is there on the table... :)
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 13758421
Documentation to backup my suggestion to first look for duplex mismatches http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00800a7af0.shtml
Assuming a mish-mash of older/newer cameras and old hubs and new switches, it never hurts to start looking at the most obvious places. Many uplinks become bottlenecks because of a port duplex mismatch between switches or between a switch and a hub, but I'll give purplepomegranite his due that this is also another obvious place to look.
Rule out issues using the 7 layer OSI model. Always start at layer 1 - a good physical inspection. Layout/topology, cable quality, loose connectors, environmental interference (cables running across florescent light fixtures), Then go to layer 2 looking at speed/duplex settings, etc..

The next place to look is port utilization. Of course old hubs can't give you that... but the uplink to the switchport can.
Do you know what type switches they have? Many have a web-based utility that can give you a good look at all port utilization as well as error counts.

Where are these cameras sending the video? To a server? Can the server handle the input? Is its CPU pegged out? Out of memory? Is it local, or across a layer 3 ip subnet boundary? What's in between the cameras and the server, if anything?

These are all just meant to get you thinking about what to look for..

I'll take that tenner bet... <8-}
ausadmin, I'm looking forward to your report on what you find!
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13758473
Good article that, lrmoore, I am also looking forward to the report back!! :)
0
 

Author Comment

by:ausadmin
ID: 13759505
Wow this is great guys - I might have a chance.

Dumb question - how much is 'excessive' in a 100 mbps environment and a 10 mbps environment (all switched) - I spoke to the client again and he thinks some of the modules in the switches may be 10.

Will ethereal show which ones are 10 - or do I just figure that out with the numbers?

And packet generation - I was thinking that one of the switches maybe struggling with large or many small packets.  Anything anyone know that can test generate this kind of traffic?
0
 

Author Comment

by:ausadmin
ID: 13759559
Last Q promise - I've got hold of 2 fluke devices - a fluke nettool and an Etherscope  - how easy are they to drive - should I sink my time into those?
0
 

Author Comment

by:ausadmin
ID: 13759623
Whoops I lied :-/

That link for the cisco switches is definately great and geting favoritised.  Their switches are 3com baseline 2226 (3C16475C)
http://www.3com.com/products/en_US/detail.jsp?tab=support&pathtype=support&sku=3C16475A

I've only ever worked with cisco and netgear.

Anyone got sofware/links on these?



0
 
LVL 23

Expert Comment

by:Tim Holman
ID: 13761050
I think it would be a good idea to verify / audit all the network equipment first, check that 100 FDX capable equipment is enabled throughout, and that any old hubs or 10 FDX equipment are phased out.

In particular, pay attention to the CCTV cameras, and individually check that they are negotiating the best possible speed with the switches.  If some CCTV cameras are old and can only run 10 HDX, then separate these out onto their own hub and uplink this hub to 100Mbs switch.

A Sniffer isn't  going to help just yet - it's more a process of verifying network design and configuration at this stage (which is exactly what I would do if someone contracted me in to do this work).

0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13761056
It sounds to me like the client doesn't really know much about his network!!  The first thing I'd be doing is charging him to give him a network diagram- any network that consists of more than ten or so network devices should really have one.  It makes adding to the system much much easier in the future.

The Fluke Nettool could well be useful.  It identifies duplex mismatching which lrmoore raised, and can analyse the traffic between points, both useful features in this case.  In fact it should pretty much identify most problems, so once you have your diagram (yeah, you've guessed it, I'm a real stickler for that!!) testing various points on the network should isolate the problem.   Don't know about the Etherscope, never used one.

In my view switches are switches.  All managed switches have a telnet interface and some a web-interface.  Once you're in that, it should be pretty straight-forward to find out what it is doing.  The switches you have linked support the web interface in the latest firmware (available here http://www.3com.com/products/en_US/result.jsp?selected=6&sort=effdt&sku=3C16490&order=desc).
0
 
LVL 27

Assisted Solution

by:pseudocyber
pseudocyber earned 500 total points
ID: 13761101
You don't need any software for these:

Features & Benefits
Affordable, Unmanaged 10/100 Ethernet Switching with Gigabit Uplinks

Fixed-configuration design makes the switch easy to install right out of the box

Pre-configured setup makes sure that switches work "right out of the box" with no configuration or management software needed

What all this marketing crap means is you CAN'T MANAGE THEM.  Make sure all the cameras are Autonegotiate, or whatever speed HALF duplex to ensure NO duplex mismatches.

lrmoore is right on with duplex mismatch errors - and I'll lay 10 down with him - just because a device is "Autosensing" doesn't mean anything if the device on the OTHER end isn't the same.  In fact if the Autosensing side doesn't detect a link negotiation pulse, it will SET ITSELF TO HALF DUPLEX BY DESIGN - so if your other side is 100 Full Duplex then you get a duplex mismatch.

Also NOTE - YOU CAN'T SNIFF ON THESE SWITCHES.  You can't mirror/span ports.  This means if you want to you ANY sniffer/packet capture device you're going to have to haul along a HUB (a real HUB, not a switch sold as a hub) and plug the camera and the uplink to the network into the hub and the sniffer into the hub.  This may change the physical characteristics of the problem so you won't really be able to observe what is happening anyway - people get what they pay for when they go with unmanaged equipment.  Grrr....

HTH
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13761168
lol, oh yeah, forget that firmware download... looks like there was a PWR on the end of the product code which is for a different switch.  My mistake, heh.

Don't need to be able to sniff on the switches though if you have the Fluke Nettool with you.  I'd rely on that, you can use it without actually changing the network (which as stated, could change or remove the problem).
0
 
LVL 27

Expert Comment

by:pseudocyber
ID: 13761971
For an EXCELLENT white paper discussing "Fixing Ethernet and Fast Ethernet Link Problems" with the first section entitled, "Auto-negotiation - how it is supposed to work" go to:

http://www.flukenetworks.com/us/_Promotions/ESV/Autonegotiation+WP+Internal+Splash.htm

You will have to fill out a little survey and register, but it's well worth it.

At the risk of "posting marketing material" ... I'm going to paste in this quote only so that questioners and other Experts can sample the depth of material covered in this white paper.  Once you read through this, you'll totally understand the duplex mismatch problem.

Paste:

Auto-negotiation – how it is supposed to work
Ethernet nodes connected together via a twisted pair cable negotiate their speed as well as duplex mode prior to establishing link. This process is called auto-negotiation and is performed by means of so-called Link Pulses. One of two types of link pulse signal is used to inform a “link partner” that a new station is on the line and that it is seeking to establish a connection.
These two versions of link pulse are Normal Link Pulse (NLP) and Fast Link Pulse (FLP).

The 10BASE-T link pulse (the NLP) consists simply of a half-wave pulse, transmitted eight times per second on the TX pair whenever data is not being sent. Data signals are from +1 to -1 volts.

When Fast Ethernet was developed, the standards body was very careful to ensure that backward compatibility be maintained. To accomplish this, they selected the simplest mechanism available for negotiating which physical signaling system and configuration to use. The FLP burst relies upon the presence or absence of NLPs in a certain order to communicate a binary word where each bit position represented a particular link ability. Fast Link Pulse bursts are implemented using NLPs, but as a burst of pulses that contains all of the information about the connecting device’s speed and duplex capabilities - this is the
FLP “link word,” which informs a link partner of what its abilities are. The presence of a “data” pulse in the FLP indicates a binary 1, while the absence indicates a binary 0. Data pulses appear between clocking pulses. There are 17 clocking pulses and the opportunity for 16 data pulses, so an FLP may have between 17 and 33 pulses.

-- end paste.

Hope this helps.
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13814961
I'm still waiting with anticipation to know whether I have won the tenner or lost it!!!
0
 
LVL 27

Expert Comment

by:pseudocyber
ID: 13814972
Purple, you've "lost it" ;D
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 13816524
lol, oh yeah... thanks for the clarification :)
0
 
LVL 27

Expert Comment

by:pseudocyber
ID: 14264749
Interested.
0
 
LVL 24

Expert Comment

by:purplepomegranite
ID: 14268420
Definitely interested... is my tenner safe or not?!
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 14268466
Not!
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article explains the fundamentals of industrial networking which ultimately is the backbone network which is providing communications for process devices like robots and other not so interesting stuff.
This month, Experts Exchange’s free Course of the Month is focused on CompTIA IT Fundamentals.
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question