Solved

Problems with Cisco switch and some of the ports

Posted on 2010-09-14
11
540 Views
Last Modified: 2012-08-14
We have a Cisco 3650 switch with a flat vlan that is linked to another Extreme switch via a fiber whip and mini-gbic. When I got to the office it appeared that the gbic on the Cisco switch had died and when I replaced the gbic the link came back up. Well when it appeared to be fine again I noticed we started losing connectivy to a couple servers, one being a VM on a Hyper-V server. You could get to the Hyper-V but not the guest. odd. Anyway it started to progress and then it was a handful of servers that could not communicate. I immidiately unplugged the uplink and the problem was still there. We traced the issue down to 2 ports that seemed to be bad. For example - When we hung a workstation off the port to test we could hit a some addresses of servers on the switch and others we couldn't. Very inconsistant. Since the issue yesterday I have left the other switch unplugged and we are not using the 2 (so called) bad ports.

What I am trying to find out is if there is a correlation between the port that failed and the issues I had with the intermittent connectivity with just SOME of the ports and VM's on the switch. I can explain this a little deeper if needed but was wondering if this may hit a nerve with someone who has had a similar experience.

 Again seemed to be when I plugged the switch back in that the problems started to creep in. It wasn't all at once but progressive. Even after I unplugged the uplink the problem was still there.
0
Comment
Question by:tobyhansen
  • 5
  • 3
  • 3
11 Comments
 
LVL 22

Accepted Solution

by:
Matt V earned 333 total points
Comment Utility
Progressive connection problems like that usually indicate a spanning tree issue.
0
 
LVL 1

Author Comment

by:tobyhansen
Comment Utility
That is what I thought but I am having a problem locating evidence and probable cause for a STP related port block. Any ideas what I should look for? I see some evidence of excessive broadcast on a few ports. Why would this just show up out of the blue?
0
 
LVL 22

Expert Comment

by:Matt V
Comment Utility
Do you have multiple NICs connected to the same switch from the Hyper-V host server?  Depending on how the virtual switching is setup, this could create a spanning tree loop.
Also, I have seen broadcom network cards create loops and steal IPs and all other sorts of non-sense when they are teamed using the Broadcom software and then used for VMware/Hyper-V/Xenserver main network links.
0
 
LVL 1

Author Comment

by:tobyhansen
Comment Utility
It could have been something with the Hyper-V server as that was the machine that had the original issues. It was very odd. You could get to the host but not 'some' of the VM's. The thing is I think they are all on the same virtual switch with just one NIC going to the physical switch.

Can these issues be progressive? Can they start small and escelate for no reason to other servers, vitual and physical?

Is there a way to tell on a Cisco switch exactly what ports may have been flagged by STP and limited or blocked? I could not find any evidence in the switch logs. I am about to turn this back on in the morning.
0
 
LVL 22

Expert Comment

by:Matt V
Comment Utility
Yes, you can turn on debugging to watch the STP events.

switch# term mon
switch# debug spantree events

Might be slighlty different debug command I have an older switch with older IOS on it.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 1

Assisted Solution

by:cdusio
cdusio earned 167 total points
Comment Utility
show spanning-tree blocked ports is what you are looking for.
or show spanning-tree vlan XXX
0
 
LVL 1

Author Comment

by:tobyhansen
Comment Utility
We ran this and didn't see any blocked ports. Yet now we got another port that seemed to go haywire.

We can ping just a couple other servers on the switch when connected to one of these failed ports but can't hit anytihg else. Very odd. Any thoughts?

Again the server on the port that goes down can no longer communicate out... except for just a couple other ports. STP doens't show any blocked ports. Now we have 3 ports that when we plug into can only see a couple IP's.. and nothing else. Not sure if I am explaining the right.
0
 
LVL 1

Expert Comment

by:cdusio
Comment Utility
any security on the switch? Can you post a sanitized version of the config?
0
 
LVL 1

Author Comment

by:tobyhansen
Comment Utility
Here you go... like I said very basic.
switch.txt
0
 
LVL 1

Expert Comment

by:cdusio
Comment Utility
That's about as basic as it gets.
could just be a bad switch.

You could check the arp table on the switch or the mac-asddress table see if something looks weird but that config is almost default.
0
 
LVL 1

Author Comment

by:tobyhansen
Comment Utility
We reset the switch and the problem so far has been eliminated. Thanks for the help.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

I see many questions here on Experts Exchange regarding switch port configurations and trunks. This article is meant for beginners in the subject to help to get basic knowledge about Virtual Local Area Network (VLAN (http://en.wikipedia.org/wiki/Vir…
This article is in response to a question (http://www.experts-exchange.com/Networking/Network_Management/Network_Analysis/Q_28230497.html) here at Experts Exchange. The Original Poster (OP) requires a utility that will accept a list of IP addresses …
This video discusses moving either the default database or any database to a new volume.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now