mmiller01
asked on
Determining cause of Overruns+Drops on Cisco switchport
We have four ports connected to an Isilon storage cluster that are getting a relatively high number of drops and the same number of overruns. Both the switch and the Isilon NIC are set to autonegotiate, and the port setup is identical to our other larger Isilon cluster that does not exhibit this issue, but has much lower bit rates on the ports. The bit rate is ten time higher on these four ports than our other cluster, so I need to determin if the ports are simply oversubscribed or if there is another issue. Equipment is a Cisco 6513 chassis with ports on 6548-ge-tx modules.
Show interface below -
Switch#show int gig 8/13
GigabitEthernet8/13 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0017.5a28.57bc (bia 0017.5a28.57bc)
Description: *** Node4 ***
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Full-duplex, 1000Mb/s
input flow-control is off, output flow-control is desired
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters 16:45:59
Input queue: 0/2000/4893/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 7314000 bits/sec, 743 packets/sec
5 minute output rate 1777000 bits/sec, 376 packets/sec
56319522 packets input, 55285682936 bytes, 0 no buffer
Received 1583 broadcasts (0 multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 4893 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
47013390 packets output, 45904381346 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
Show interface below -
Switch#show int gig 8/13
GigabitEthernet8/13 is up, line protocol is up (connected)
Hardware is C6k 1000Mb 802.3, address is 0017.5a28.57bc (bia 0017.5a28.57bc)
Description: *** Node4 ***
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Full-duplex, 1000Mb/s
input flow-control is off, output flow-control is desired
Clock mode is auto
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:00, output hang never
Last clearing of "show interface" counters 16:45:59
Input queue: 0/2000/4893/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 7314000 bits/sec, 743 packets/sec
5 minute output rate 1777000 bits/sec, 376 packets/sec
56319522 packets input, 55285682936 bytes, 0 no buffer
Received 1583 broadcasts (0 multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 4893 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
47013390 packets output, 45904381346 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I tried hard setting the speed/dup on both ends and no change.
I moved the ports to a 16 port module (6516-ge-tx) that has a different buffer structure and have not dropped a packet since. something about the 48 port module buffers that cannot hack that volume it seems.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
It seems moving to the 16 port modules has eliminated the drops.
Going through other ports on the 48 port modules I do not see any drops as well. Odd that something about this traffic was causing them.
At this point I can call the initial issue fixed.
Going through other ports on the 48 port modules I do not see any drops as well. Odd that something about this traffic was causing them.
At this point I can call the initial issue fixed.
Common Cause: The input rate of traffic exceeded the ability of the receiver to handle the data.
Could have been a sustained spike in traffic? Do you have MRTG or cricket to monitir your interfaces?
harbor235 ;}