• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 892
  • Last Modified:

Using Ethereal to pinpoint a network throughput problem

Hi All

Getting sent out tomorrow to do some network analysis which I've only done a very little of.

Some rental place gave me an ancient version of sniffer pro 3.0.5 on a dinosaur laptop.  


The problem is that the client has a number of CCTV cameras mainly on 100 mb switches but there may be some 10s or 10 links.

When they up the cameras to 25 frames per second they get dropouts - in fact get dropouts at anything above 6.  They've got about 25 cameras and they said they've done 18 camera installs before on 100mbs networks - no worries.

So a few Qs: Pointing me to the right bits of the manual would be peachy
(oh and I have to operate in a strictly windows environment)


- Does ethereal handle switched networks and if so do I have to set anything special up.

- Can ethereal generate traffic and do throughput testing if so - how? and if not do people know of any other free tools I can use.  There used to be this thing called Raccoon speed tester that did timed transfers but I can't find it anymore on the web.  I'm guessing if it gets down to some switch not handling the type of packets I'll need something that can throw different packet types.

- Reporting - the client wants the problem pin-pointed and proved on paper.

- General tips and gotchas

- Should I send the sniffer pro back - is it going to give me anything important in this scenario that ethereal won't.

Sorry for all the Qs but I've been chucked in the deep end on this one and one of our techs has already hit this and bounced so it's up to me to save the day  :(


Much TIA
0
ausadmin
Asked:
ausadmin
  • 9
  • 4
  • 4
  • +3
4 Solutions
 
purplepomegraniteCommented:
Ouch, I don't envy you!!

Ethereal will give you an insight into everything that passes the point in the network to which you connect.  It works by watching the network in promiscuous mode (i.e. captures all packets on the interface).  My guess is that it is only going to be useful in this situation if you find the right place to seat it.

It sounds to me that the network you are going to check has a bottleneck.  If it consists of many switches interconnected, the bottleneck is likely to be between these.  e.g. If you have a 100M switch with four devices hanging off it, but this switch is uplinked to another switch further up the line via one link, then each of the four devices connected to the first switch will only have an effective bandwidth of 25Mbps.  If switches are daisychained this is reduced dramatically.

I would start by getting an accurate network diagram - you will be able to work out any such bottle necks from this and may well not even need tools such as Ethereal.
0
 
mikebernhardtCommented:
Many switches allow you to mirror traffic from one port to another, or from a group of ports to another. You'll need to do this to see any traffic on a switch.

I think the main thing you need to do is identify what percentage of bandwidth is camera and what is other things. Just because they've done 18 cameras elsewhere doesn't mean they can do 25 cameras here- it depends on the topology as purple said, and also on what else is using bandwidth. How much bandwidth is each camera generating? They should be able to tell you that. Or you can figure it out if you know image resolution and type of compression- there are tables out there that will give you bandwidth requirements.

25 cameras at 25 frames/sec could easily generate more than 100mb/sec traffic- and you can only expect a 100mb link to support about 60mb/sec without beginning to see a lot of congestion- indicated by dropped frames.
0
 
lrmooreCommented:
I would bet that the primary issue you are going to find is duplex mismatches. You can easily find this by looking at port error counts on each switch port.
Problem in a nutshell:
Camera is only capable of 10M half-duplex, switchport set to 100/full-duplex = no connection
Camera is auto-negotiate 10/100 half/full, switchport set to 100-full = duplex mismatch
Camera is set to 10 half, switchport set to 10/full = duplex mismatch
Camera is set to 10/full, switchport set to auto = duplex mismatch
Camera is set to 100/full, switchport set to auto = duplex mismatch
Camera is set to 10/full, connected to 10Mb hub = duplex mismatch
Camera is set to either 10 or 100/full connected to 10/100 HUB = duplex mismatch
Camera is set to auto, switchport set to auto = OK
Camera is set to auto, connect to HUB = OK
Camera is set to 100/full, switchport set to 100/full = OK
Camera is set to 10/full, switchport set to 10/full = OK
Camera is set to 10/half, switchport set to 10/half = OK
Camera is set to 10/half, connect to hub = OK

Types of errors that you would look for are CRC errors, excessive collisions especially if the port is set to full-duplex..
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
purplepomegraniteCommented:
Heh, I'll bet £10 it's not duplex ;)  Most switches are autosensing, esp. 10/100 switches.
0
 
Tim HolmanCommented:
Would agree with lrmoore - duplex issues are almost certainly the problem here, so you would need to log onto each switch and look at the port status.  The lights at the front may even help out - on a cisco they go orange for half-duplex and green for full duplex.  You want them all to be full duplex.
Anyway, to answer your questions:

- Does ethereal handle switched networks and if so do I have to set anything special up.

You need to setup a SPAN port on the switch, so that all traffic from all ports goes via the PC you've installed Ethereal on.

- Can ethereal generate traffic and do throughput testing if so - how? and if not do people know of any other free tools I can use.  There used to be this thing called Raccoon speed tester that did timed transfers but I can't find it anymore on the web.  I'm guessing if it gets down to some switch not handling the type of packets I'll need something that can throw different packet types.

Sort of.  An Ethereal capture will display usage summary, but it's only good for the time you capture, and to do throughput testing you really need to leave something running for a good 24 hours (at which point your Ethereal laptop will be full up and may keel over... worth a try though!).

- Reporting - the client wants the problem pin-pointed and proved on paper.

I suspect he'd want it fixed first!  You can document what you discovered, but that's about it.

- General tips and gotchas

You'll find these out as you go along... ;)

- Should I send the sniffer pro back - is it going to give me anything important in this scenario that ethereal won't.

No.  Ethereal is fine for point to point traffic capturing.  Sniffer Pro is better to use on Enterprise networks when you need to correlate data from several different network monitors.
0
 
lrmooreCommented:
>- Should I send the sniffer pro back
I wouldn't. The Expert analysis module is worth its weight in gold... Gives Symptoms/diagnoses
0
 
purplepomegraniteCommented:
Ok, my tenner is there on the table... :)
0
 
lrmooreCommented:
Documentation to backup my suggestion to first look for duplex mismatches http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00800a7af0.shtml
Assuming a mish-mash of older/newer cameras and old hubs and new switches, it never hurts to start looking at the most obvious places. Many uplinks become bottlenecks because of a port duplex mismatch between switches or between a switch and a hub, but I'll give purplepomegranite his due that this is also another obvious place to look.
Rule out issues using the 7 layer OSI model. Always start at layer 1 - a good physical inspection. Layout/topology, cable quality, loose connectors, environmental interference (cables running across florescent light fixtures), Then go to layer 2 looking at speed/duplex settings, etc..

The next place to look is port utilization. Of course old hubs can't give you that... but the uplink to the switchport can.
Do you know what type switches they have? Many have a web-based utility that can give you a good look at all port utilization as well as error counts.

Where are these cameras sending the video? To a server? Can the server handle the input? Is its CPU pegged out? Out of memory? Is it local, or across a layer 3 ip subnet boundary? What's in between the cameras and the server, if anything?

These are all just meant to get you thinking about what to look for..

I'll take that tenner bet... <8-}
ausadmin, I'm looking forward to your report on what you find!
0
 
purplepomegraniteCommented:
Good article that, lrmoore, I am also looking forward to the report back!! :)
0
 
ausadminAuthor Commented:
Wow this is great guys - I might have a chance.

Dumb question - how much is 'excessive' in a 100 mbps environment and a 10 mbps environment (all switched) - I spoke to the client again and he thinks some of the modules in the switches may be 10.

Will ethereal show which ones are 10 - or do I just figure that out with the numbers?

And packet generation - I was thinking that one of the switches maybe struggling with large or many small packets.  Anything anyone know that can test generate this kind of traffic?
0
 
ausadminAuthor Commented:
Last Q promise - I've got hold of 2 fluke devices - a fluke nettool and an Etherscope  - how easy are they to drive - should I sink my time into those?
0
 
ausadminAuthor Commented:
Whoops I lied :-/

That link for the cisco switches is definately great and geting favoritised.  Their switches are 3com baseline 2226 (3C16475C)
http://www.3com.com/products/en_US/detail.jsp?tab=support&pathtype=support&sku=3C16475A

I've only ever worked with cisco and netgear.

Anyone got sofware/links on these?



0
 
Tim HolmanCommented:
I think it would be a good idea to verify / audit all the network equipment first, check that 100 FDX capable equipment is enabled throughout, and that any old hubs or 10 FDX equipment are phased out.

In particular, pay attention to the CCTV cameras, and individually check that they are negotiating the best possible speed with the switches.  If some CCTV cameras are old and can only run 10 HDX, then separate these out onto their own hub and uplink this hub to 100Mbs switch.

A Sniffer isn't  going to help just yet - it's more a process of verifying network design and configuration at this stage (which is exactly what I would do if someone contracted me in to do this work).

0
 
purplepomegraniteCommented:
It sounds to me like the client doesn't really know much about his network!!  The first thing I'd be doing is charging him to give him a network diagram- any network that consists of more than ten or so network devices should really have one.  It makes adding to the system much much easier in the future.

The Fluke Nettool could well be useful.  It identifies duplex mismatching which lrmoore raised, and can analyse the traffic between points, both useful features in this case.  In fact it should pretty much identify most problems, so once you have your diagram (yeah, you've guessed it, I'm a real stickler for that!!) testing various points on the network should isolate the problem.   Don't know about the Etherscope, never used one.

In my view switches are switches.  All managed switches have a telnet interface and some a web-interface.  Once you're in that, it should be pretty straight-forward to find out what it is doing.  The switches you have linked support the web interface in the latest firmware (available here http://www.3com.com/products/en_US/result.jsp?selected=6&sort=effdt&sku=3C16490&order=desc).
0
 
pseudocyberCommented:
You don't need any software for these:

Features & Benefits
Affordable, Unmanaged 10/100 Ethernet Switching with Gigabit Uplinks

Fixed-configuration design makes the switch easy to install right out of the box

Pre-configured setup makes sure that switches work "right out of the box" with no configuration or management software needed

What all this marketing crap means is you CAN'T MANAGE THEM.  Make sure all the cameras are Autonegotiate, or whatever speed HALF duplex to ensure NO duplex mismatches.

lrmoore is right on with duplex mismatch errors - and I'll lay 10 down with him - just because a device is "Autosensing" doesn't mean anything if the device on the OTHER end isn't the same.  In fact if the Autosensing side doesn't detect a link negotiation pulse, it will SET ITSELF TO HALF DUPLEX BY DESIGN - so if your other side is 100 Full Duplex then you get a duplex mismatch.

Also NOTE - YOU CAN'T SNIFF ON THESE SWITCHES.  You can't mirror/span ports.  This means if you want to you ANY sniffer/packet capture device you're going to have to haul along a HUB (a real HUB, not a switch sold as a hub) and plug the camera and the uplink to the network into the hub and the sniffer into the hub.  This may change the physical characteristics of the problem so you won't really be able to observe what is happening anyway - people get what they pay for when they go with unmanaged equipment.  Grrr....

HTH
0
 
purplepomegraniteCommented:
lol, oh yeah, forget that firmware download... looks like there was a PWR on the end of the product code which is for a different switch.  My mistake, heh.

Don't need to be able to sniff on the switches though if you have the Fluke Nettool with you.  I'd rely on that, you can use it without actually changing the network (which as stated, could change or remove the problem).
0
 
pseudocyberCommented:
For an EXCELLENT white paper discussing "Fixing Ethernet and Fast Ethernet Link Problems" with the first section entitled, "Auto-negotiation - how it is supposed to work" go to:

http://www.flukenetworks.com/us/_Promotions/ESV/Autonegotiation+WP+Internal+Splash.htm

You will have to fill out a little survey and register, but it's well worth it.

At the risk of "posting marketing material" ... I'm going to paste in this quote only so that questioners and other Experts can sample the depth of material covered in this white paper.  Once you read through this, you'll totally understand the duplex mismatch problem.

Paste:

Auto-negotiation – how it is supposed to work
Ethernet nodes connected together via a twisted pair cable negotiate their speed as well as duplex mode prior to establishing link. This process is called auto-negotiation and is performed by means of so-called Link Pulses. One of two types of link pulse signal is used to inform a “link partner” that a new station is on the line and that it is seeking to establish a connection.
These two versions of link pulse are Normal Link Pulse (NLP) and Fast Link Pulse (FLP).

The 10BASE-T link pulse (the NLP) consists simply of a half-wave pulse, transmitted eight times per second on the TX pair whenever data is not being sent. Data signals are from +1 to -1 volts.

When Fast Ethernet was developed, the standards body was very careful to ensure that backward compatibility be maintained. To accomplish this, they selected the simplest mechanism available for negotiating which physical signaling system and configuration to use. The FLP burst relies upon the presence or absence of NLPs in a certain order to communicate a binary word where each bit position represented a particular link ability. Fast Link Pulse bursts are implemented using NLPs, but as a burst of pulses that contains all of the information about the connecting device’s speed and duplex capabilities - this is the
FLP “link word,” which informs a link partner of what its abilities are. The presence of a “data” pulse in the FLP indicates a binary 1, while the absence indicates a binary 0. Data pulses appear between clocking pulses. There are 17 clocking pulses and the opportunity for 16 data pulses, so an FLP may have between 17 and 33 pulses.

-- end paste.

Hope this helps.
0
 
purplepomegraniteCommented:
I'm still waiting with anticipation to know whether I have won the tenner or lost it!!!
0
 
pseudocyberCommented:
Purple, you've "lost it" ;D
0
 
purplepomegraniteCommented:
lol, oh yeah... thanks for the clarification :)
0
 
pseudocyberCommented:
Interested.
0
 
purplepomegraniteCommented:
Definitely interested... is my tenner safe or not?!
0
 
lrmooreCommented:
Not!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

  • 9
  • 4
  • 4
  • +3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now