Solved

Fiber optic interface going down

Posted on 2010-09-02
12
1,732 Views
Last Modified: 2012-05-10
We have two Foundry switches connected via two single mode fiber ports between two sites located about a mile apart.  We are using long range SFPs E1MG-LX which are supposed to be good up to 10km with SMF.

Port 1 is connected to Port 1 and Port 2 is connected to Port 2.

Sw 1 is the root bridge because it has a lower mac address xxxx001bed6c1760 than Sw 2 xxxx001bed6c1900.

We think that the fiber optic cabling between Port 1 is bad but I am wondering why Port 1 and Port 2 on Sw 1 are going through all the STP port states?

Shouldn't Port 1 and Port 2 on the root switch always be designated and in forwarding mode?

We currently have Port 1 on Sw 1 disabled to force Port 2 to pass the traffic. We also enabled UDLD because even with two redundant paths we were losing connectivity between these two switches.

Sorry the log only has up time.

I tried to match up the two logs and they don't seem to match.

Does this look like a physical issue?

Sw 1 log
256d00h28m13s:I:startup-config was changed by  from telnet client xxx.16.108.2
256d00h27m01s:I:startup-config was changed by  from telnet client xxx.16.108.2
256d00h27m00s:I:running-config was changed by  from telnet client xxx.16.108.2
256d00h26m29s:I:PORT: 1 disabled by  from telnet session
256d00h26m28s:I:System: Interface ethernet 1, state down
256d00h26m28s:I:STP: VLAN 108 Port 1 STP State -> BLOCKING (DOT1wTransition)
256d00h26m28s:I:STP: VLAN 108 Port 1 STP State -> FORWARDING (PortDown)
256d00h25m48s:I:STP: VLAN 108 Port 2 Bridge TC Event (DOT1wTransition)
256d00h25m48s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (DOT1wTransition)
256d00h25m33s:I:STP: VLAN 108 Port 2 STP State -> LEARNING (DOT1wTransition)
256d00h25m24s:I:running-config was changed by  from telnet client xxx.16.108.2
256d00h25m19s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
255d23h13m19s:I:running-config was changed by  from telnet client xxx.16.108.2
255d23h12m48s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (DOT1wTransition)
255d23h12m48s:I:STP: VLAN 108 Port 2 STP State -> LEARNING (DOT1wTransition)
255d23h12m45s:I:System: Logical link on interface ethernet 2 is up.
255d23h12m45s:I:System: Interface ethernet 2, state up
255d23h12m45s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
255d23h12m32s:I:System: Logical link on interface ethernet 2 is down.
255d23h12m32s:I:System: Interface ethernet 2, state down
255d23h12m32s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
255d23h12m32s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (PortDown)
255d23h10m18s:I:running-config was changed by  from telnet client xxx.16.108.2
255d21h51m17s:I:STP: VLAN 108 Port 1 Bridge TC Event (DOT1wTransition)
255d21h51m17s:I:STP: VLAN 108 Port 1 STP State -> FORWARDING (DOT1wTransition)
255d21h51m17s:I:STP: VLAN 108 Port 1 STP State -> LEARNING (DOT1wTransition)
255d21h51m17s:I:System: Interface ethernet 1, state up
255d21h51m17s:I:STP: VLAN 108 Port 1 STP State -> BLOCKING (DOT1wTransition)
255d18h07m25s:I:STP: VLAN 108 Port 2 Bridge TC Event (DOT1wTransition)
255d18h07m25s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (DOT1wTransition)
255d18h07m25s:I:STP: VLAN 108 Port 2 STP State -> LEARNING (DOT1wTransition)
255d18h07m24s:I:System: Interface ethernet 2, state up
255d18h07m24s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
255d18h06m43s:I:System: Interface ethernet 2, state down
255d18h06m43s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
255d18h06m43s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (PortDown)
255d18h06m39s:I:System: Interface ethernet 1, state down
255d18h06m39s:I:STP: VLAN 108 Port 1 STP State -> BLOCKING (DOT1wTransition)
255d18h06m39s:I:STP: VLAN 108 Port 1 STP State -> FORWARDING (PortDown)

Sw 2
263d00h22m14s:I:startup-config was changed by  from telnet client xxx.16.108.4
263d00h21m53s:I:startup-config was changed by  from telnet client xxx16.108.4
263d00h21m49s:I:running-config was changed by  from telnet client xxx.16.108.4
263d00h20m47s:I:System: Interface ethernet 1, state down
263d00h20m47s:I:STP: VLAN 108 Port 1 STP State -> BLOCKING (DOT1wTransition)
263d00h20m47s:I:STP: VLAN 108 Port 2 Bridge TC Event (DOT1wTransition)
263d00h20m47s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (DOT1wTransition)
263d00h20m47s:I:STP: VLAN 108 Port 2 STP State -> LEARNING (DOT1wTransition)
263d00h20m47s:I:STP: VLAN 108 New RootPort: 2 (RootSelection)
263d00h20m47s:I:STP: VLAN 108 Port 1 STP State -> FORWARDING (PortDown)
263d00h19m38s:I:System: Logical link on interface ethernet 2 is up.
263d00h19m38s:I:System: Interface ethernet 2, state up
263d00h19m38s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
263d00h19m09s:I:running-config was changed by  from telnet client xxx.16.108.2
263d00h19m04s:I:System: Logical link on interface ethernet 2 is down.
263d00h19m04s:I:System: Interface ethernet 2, state down
263d00h19m04s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
263d00h19m04s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (PortDown)
262d23h06m49s:I:running-config was changed by  from telnet client xxx.16.108.4
262d23h04m22s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d23h04m21s:I:System: Logical link on interface ethernet 2 is up.
262d23h04m21s:I:System: Interface ethernet 2, state up
262d23h04m21s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d23h03m05s:I:running-config was changed by  from telnet client xxx.16.108.2
262d23h02m41s:I:System: Logical link on interface ethernet 2 is down.
262d23h02m41s:I:System: Interface ethernet 2, state down
262d23h02m41s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d23h02m41s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (PortDown)
262d21h45m37s:I:STP: VLAN 108 Port 1 Bridge TC Event (DOT1wTransition)
262d21h45m37s:I:STP: VLAN 108 Port 1 STP State -> FORWARDING (DOT1wTransition)
262d21h45m37s:I:STP: VLAN 108 Port 1 STP State -> LEARNING (DOT1wTransition)
262d21h45m37s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d21h45m37s:I:STP: VLAN 108 New RootPort: 1 (RootSelection)
262d21h45m37s:I:System: Interface ethernet 1, state up
262d21h45m37s:I:STP: VLAN 108 Port 1 STP State -> BLOCKING (DOT1wTransition)
262d21h26m44s:I:running-config was changed by  from telnet client xxx.16.108.4
262d21h26m26s:I:PORT: 1 enabled by  from telnet session
262d21h25m47s:I:running-config was changed by  from telnet client xxx.16.108.4
262d21h25m42s:I:PORT: 1 disabled by  from telnet session
262d18h01m44s:I:STP: VLAN 108 New RootBridge: xxxx001bed6c1760 RootPort: 2 (BpduRcvd)
262d18h01m44s:I:STP: VLAN 108 Port 2 Bridge TC Event (DOT1wTransition)
262d18h01m44s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (DOT1wTransition)
262d18h01m44s:I:STP: VLAN 108 Port 2 STP State -> LEARNING (DOT1wTransition)
262d18h01m44s:I:STP: VLAN 108 New RootPort: 2 (RootSelection)
262d18h01m43s:I:System: Interface ethernet 2, state up
262d18h01m43s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d18h01m02s:I:System: Interface ethernet 2, state down
262d18h01m02s:I:STP: VLAN 108 Bridge is RootBridge: xxxx001bed6c1900 (PortDown)
262d18h01m02s:I:STP: VLAN 108 Port 2 STP State -> BLOCKING (DOT1wTransition)
262d18h01m02s:I:STP: VLAN 108 Port 2 STP State -> FORWARDING (PortDown)
0
Comment
Question by:Dragon0x40
  • 5
  • 4
  • 2
  • +1
12 Comments
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Can you try bringing both interfaces down at the same time and then up in different order to see if it helps?
Foundry is bought by brocade, so you can call them for "virtual cable tester" in firmware in your device.
0
 
LVL 28

Accepted Solution

by:
mikebernhardt earned 167 total points
Comment Utility
It does look like a physical issue, because the actual links are dropping and returning. If it was a configuration issue the ports would most likely stay up. It could even be a dirty connector or patch somewhere.

You should change the log timestamps to clock time instead of system uptime, and set them both to use NTP. Then you would have accurate timestamps.
0
 
LVL 24

Assisted Solution

by:rfc1180
rfc1180 earned 333 total points
Comment Utility
>two single mode fiber ports between two sites located about a mile apart.
You can get the carrier involved to test the fiber; it does sound like you have a layer 1 issue. I would clean the fiber, possibly replace the SFP modules on both ends if the carrier comes back and states all is well.
>Shouldn't Port 1 and Port 2 on the root switch always be designated and in forwarding mode?
yes

Billy
0
 

Author Comment

by:Dragon0x40
Comment Utility
Thanks gheist,
The plan is to test the fiber between Port 1 and replace patch cables and gbics.

Thanks mikebernhardt,
We can't reach our timeservers from this subnet.

Thanks rfc1180,
Why are any ports on Sw1 ever going into blocking? I would think since SW 1 is the root that once the switch is powered up and forwarding then RSTP would not have a reason to block the designated ports?
0
 
LVL 24

Expert Comment

by:rfc1180
Comment Utility
>Why are any ports on Sw1 ever going into blocking?
regardless the protocol is RSTP, there are still some rules that apply for 802.1D, a port will always intiailize to blocking first. It is a state that needs to be initialized by the protocol (802.1D and RSTP).

Billy
0
 

Author Comment

by:Dragon0x40
Comment Utility
I agree with your statement.

I just don't get why the root switch would need to initialize?

Are you thinking that both Port 1 and Port 2 links are flapping and that is why the ports are initializing?

I was assuming that the Port 2 links were not flapping and then if RSTP needs to break the loop it would block/transition Port 2 on the non-root switch but Port 2 on the root switch should not change?
0
Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

 

Author Comment

by:Dragon0x40
Comment Utility
If both of the ports are flapping then I need to change my repair plan.
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
...We can't reach our timeservers from this subnet...

Buy a $50 usb GPS for each site ;)
0
 
LVL 24

Assisted Solution

by:rfc1180
rfc1180 earned 333 total points
Comment Utility
>I just don't get why the root switch would need to initialize?
Think about it this way, the rules state that a port (even a port that is on the root bridge) needs to initialize to block during the state transitions to ensure that the network is loop free and also there could be a switch that was added to the network that has superior BPDUs. Remember, all switches that power up on a network thinks they are root until they are all converged.

>Are you thinking that both Port 1 and Port 2 links are flapping and that is why the ports are initializing?
I can see in the logs that both ports 1 and 2 are flapping;l was that you doing something during the issue?

>I was assuming that the Port 2 links were not flapping and then if RSTP needs to break the loop it would block/transition Port 2 on the non-root switch but Port 2 on the root switch should not change?

technically, with RSTP, there is a role called Alternate port in that if a links role is an alternate path to root and if another port that is an active root port role changes due to a port dropping then the port that is alternate immediately transitions to the forwarding state skipping all other states, so with your statement "Port 2 on the root switch should not change?" that is correct, the port should stay active as designated forwarding as long as the that port is stable.

Billy
0
 

Author Comment

by:Dragon0x40
Comment Utility
thanks gheist,

GPS has a timeserver? I thought GPS was for location?

Thanks rfc1180,

With port 1 disabled, port 2 has been up without incident for 4 or 5 days. I am thinking about a TAC call to Foundry/Brocade as this may not be a cabling/physical link issue. We did add udld to try and get the failover to work cleaner but I would like to prevent any more downtime.
0
 

Author Comment

by:Dragon0x40
Comment Utility
We replaced the SFPs on both ends along with patch cables.

Tested the fiber and it showed good light level test and passed a data pattern test.

The links have not gone down since we did the work so it seems to have fixed the problem.

I will award the points shortly.
0
 
LVL 24

Expert Comment

by:rfc1180
Comment Utility
>GPS has a timeserver? I thought GPS was for location?
http://www.ntp-time-server.com/time-server/gps-ntp-server.html
0

Featured Post

Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

Join & Write a Comment

Suggested Solutions

The Cisco RV042 router is a popular small network interfacing device that is often used as an internet gateway. Network administrators need to get at the management interface to make settings, change passwords, etc. This access is generally done usi…
Tired of waiting for your show or movie to load?  Are buffering issues a constant problem with your internet connection?  Check this article out to see if these simple adjustments are the solution for you.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

728 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now