Solved

Problems with Cisco switch and some of the ports

Posted on 2010-09-14
11
542 Views
Last Modified: 2012-08-14
We have a Cisco 3650 switch with a flat vlan that is linked to another Extreme switch via a fiber whip and mini-gbic. When I got to the office it appeared that the gbic on the Cisco switch had died and when I replaced the gbic the link came back up. Well when it appeared to be fine again I noticed we started losing connectivy to a couple servers, one being a VM on a Hyper-V server. You could get to the Hyper-V but not the guest. odd. Anyway it started to progress and then it was a handful of servers that could not communicate. I immidiately unplugged the uplink and the problem was still there. We traced the issue down to 2 ports that seemed to be bad. For example - When we hung a workstation off the port to test we could hit a some addresses of servers on the switch and others we couldn't. Very inconsistant. Since the issue yesterday I have left the other switch unplugged and we are not using the 2 (so called) bad ports.

What I am trying to find out is if there is a correlation between the port that failed and the issues I had with the intermittent connectivity with just SOME of the ports and VM's on the switch. I can explain this a little deeper if needed but was wondering if this may hit a nerve with someone who has had a similar experience.

 Again seemed to be when I plugged the switch back in that the problems started to creep in. It wasn't all at once but progressive. Even after I unplugged the uplink the problem was still there.
0
Comment
Question by:tobyhansen
  • 5
  • 3
  • 3
11 Comments
 
LVL 22

Accepted Solution

by:
Matt V earned 333 total points
ID: 33676298
Progressive connection problems like that usually indicate a spanning tree issue.
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33676392
That is what I thought but I am having a problem locating evidence and probable cause for a STP related port block. Any ideas what I should look for? I see some evidence of excessive broadcast on a few ports. Why would this just show up out of the blue?
0
 
LVL 22

Expert Comment

by:Matt V
ID: 33676406
Do you have multiple NICs connected to the same switch from the Hyper-V host server?  Depending on how the virtual switching is setup, this could create a spanning tree loop.
Also, I have seen broadcom network cards create loops and steal IPs and all other sorts of non-sense when they are teamed using the Broadcom software and then used for VMware/Hyper-V/Xenserver main network links.
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33676997
It could have been something with the Hyper-V server as that was the machine that had the original issues. It was very odd. You could get to the host but not 'some' of the VM's. The thing is I think they are all on the same virtual switch with just one NIC going to the physical switch.

Can these issues be progressive? Can they start small and escelate for no reason to other servers, vitual and physical?

Is there a way to tell on a Cisco switch exactly what ports may have been flagged by STP and limited or blocked? I could not find any evidence in the switch logs. I am about to turn this back on in the morning.
0
 
LVL 22

Expert Comment

by:Matt V
ID: 33677106
Yes, you can turn on debugging to watch the STP events.

switch# term mon
switch# debug spantree events

Might be slighlty different debug command I have an older switch with older IOS on it.
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 1

Assisted Solution

by:cdusio
cdusio earned 167 total points
ID: 33694867
show spanning-tree blocked ports is what you are looking for.
or show spanning-tree vlan XXX
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33695551
We ran this and didn't see any blocked ports. Yet now we got another port that seemed to go haywire.

We can ping just a couple other servers on the switch when connected to one of these failed ports but can't hit anytihg else. Very odd. Any thoughts?

Again the server on the port that goes down can no longer communicate out... except for just a couple other ports. STP doens't show any blocked ports. Now we have 3 ports that when we plug into can only see a couple IP's.. and nothing else. Not sure if I am explaining the right.
0
 
LVL 1

Expert Comment

by:cdusio
ID: 33695896
any security on the switch? Can you post a sanitized version of the config?
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33696016
Here you go... like I said very basic.
switch.txt
0
 
LVL 1

Expert Comment

by:cdusio
ID: 33696146
That's about as basic as it gets.
could just be a bad switch.

You could check the arp table on the switch or the mac-asddress table see if something looks weird but that config is almost default.
0
 
LVL 1

Author Comment

by:tobyhansen
ID: 33725120
We reset the switch and the problem so far has been eliminated. Thanks for the help.
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Have you ever set up your wireless router at home or in the office to find that you little pop-up bubble in the bottom right-hand corner of Windows read "IP Conflict - One of more computers on the network have been assigned the following IP address"…
I eventually solved a perplexing problem setting up telnet for a new switch.  I installed a new Cisco WS-03560X-24P switch connected to an existing Cisco 4506 running a WS-X4013-10GE Sup II-Plus. After configuring vlans and trunking,  I could no…
Migrating to Microsoft Office 365 is becoming increasingly popular for organizations both large and small. If you have made the leap to Microsoft’s cloud platform, you know that you will need to create a corporate email signature for your Office 365…
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to go…

912 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

28 Experts available now in Live!

Get 1:1 Help Now