Cisco MPLS Configuration on Router Crashes Connection

Hello Experts,

I have configured MPLS between Hong Kong to London_Nr1. After enabling the interfaces gig 1/2.1 on hong kong and gig 0/1/3.24 on London_Nr1 to enable MPLS traffic I get the following error messages in the logs and I'm suddenly unable to telnet into London_Nr1.

On Hong Kong I get the following error messgaes in the show logg

Sep  2 08:26:07: %SYS-5-CONFIG_I: Configured from console by ciscodcn on vty0 (172.24.12.112)
Sep  2 08:26:19: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 1): interrupted--recovery pending
Sep  2 08:26:19: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (0) is DOWN (Interface not operational)
Sep  2 08:29:19: %BGP-5-ADJCHANGE: neighbor 172.22.16.1 Down BGP Notification sent
Sep  2 08:29:19: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.1 4/0 (hold time expired) 0 bytes
Sep  2 08:32:20: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (1) is UP
Sep  2 08:32:26: %BGP-5-ADJCHANGE: neighbor 172.22.16.1 Up
Sep  2 08:32:59: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 1): interrupted--recovery pending
Sep  2 08:32:59: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (0) is DOWN (Interface not operational)
Sep  2 08:35:58: %BGP-5-ADJCHANGE: neighbor 172.22.16.1 Down BGP Notification sent
Sep  2 08:35:58: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.1 4/0 (hold time expired) 0 bytes
Sep  2 08:37:06: %SYS-5-CONFIG_I: Configured from console by ciscodcn on vty0 (172.24.12.112)
Sep  2 08:37:42: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (1) is UP
Sep  2 08:37:44: %BGP-5-ADJCHANGE: neighbor 172.22.16.1 Up

You will notice that everything comes up on Hong Kong.

On London_Nr1

Sep  2 08:21:08.147 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.3 Up
Sep  2 08:23:06.284 GMT: %SNMP-3-AUTHFAIL: Authentication failure for SNMP req from host 93.187.32.50
Sep  2 08:24:00.446 GMT: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 1): interrupted--recovery pending
Sep  2 08:24:00.446 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (0) is DOWN (Received error notification from peer: KeepAlive expired)
Sep  2 08:26:33.652 GMT: %LDP-5-GR: GR session 172.22.16.3:0 (inst. 2): interrupted--recovery pending
Sep  2 08:26:33.652 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.3:0 (0) is DOWN (Discovery Hello Hold Timer expired)
Sep  2 08:26:41.475 GMT: %OSPF-4-CONFLICTING_LSAID: LSA origination prevented by existing LSA with same LSID but a different mask
Sep  2 08:28:07.123 GMT: %SNMP-3-AUTHFAIL: Authentication failure for SNMP req from host 93.187.32.50
Sep  2 08:29:08.131 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.3 Down BGP Notification sent
Sep  2 08:29:08.131 GMT: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.3 4/0 (hold time expired) 0 bytes Sep  2 08:29:37.178 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (1) is UP
Sep  2 08:29:37.190 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC UP, VC state UP
Sep  2 08:32:21.003 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.3:0 (2) is UP
Sep  2 08:32:26.354 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.3 Up
Sep  2 08:33:08.294 GMT: %SNMP-3-AUTHFAIL: Authentication failure for SNMP req from host 93.187.32.50
Sep  2 08:33:12.709 GMT: %LDP-5-GR: GR session 172.22.16.3:0 (inst. 2): interrupted--recovery pending
Sep  2 08:33:12.709 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.3:0 (0) is DOWN (Discovery Hello Hold Timer expired)
Sep  2 08:35:20.944 GMT: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 1): interrupted--recovery pending
Sep  2 08:35:20.944 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (0) is DOWN (Received error notification from peer: KeepAlive expired)
Sep  2 08:35:20.944 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC DOWN, VC state DOWN
Sep  2 08:35:26.503 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.3 Down BGP Notification sent
Sep  2 08:35:26.503 GMT: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.3 4/0 (hold time expired) 0 bytes
Sep  2 08:36:08.547 GMT: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 2): starting graceful recovery
Sep  2 08:36:08.547 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (2) is UP
Sep  2 08:36:09.087 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC UP, VC state UP
Sep  2 08:36:09.087 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC UP, VC state UP
Sep  2 08:36:09.087 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC UP, VC state UP
Sep  2 08:37:42.982 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.3:0 (3) is UP
Sep  2 08:37:44.281 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.3 Up
Sep  2 08:38:08.525 GMT: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 2): completed graceful recovery
Sep  2 08:39:55.173 GMT: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 2): interrupted--recovery pending
Sep  2 08:39:55.173 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (0) is DOWN (Received error notification from peer: KeepAlive expired)
Sep  2 08:39:55.173 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.2 vcid 200, VC DOWN, VC state DOWN

On the London_Nr2

Sep  2 08:24:00.363 GMT: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 1): interrupted--recovery pending
Sep  2 08:24:00.363 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (0) is DOWN (Session KeepAlive Timer expired)
Sep  2 08:24:00.363 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC DOWN, VC state DOWN
Sep  2 08:24:09.004 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.255 Down BGP Notification sent
Sep  2 08:24:09.004 GMT: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.255 4/0 (hold time expired) 0 bytes
Sep  2 08:29:37.173 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (1) is UP
Sep  2 08:29:37.193 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC UP, VC state UP
Sep  2 08:29:39.673 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.255 Up
Sep  2 08:35:20.940 GMT: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 1): interrupted--recovery pending
Sep  2 08:35:20.944 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (0) is DOWN (Session KeepAlive Timer expired)
Sep  2 08:35:20.944 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC DOWN, VC state DOWN
Sep  2 08:35:27.049 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.255 Down BGP Notification sent
Sep  2 08:35:27.049 GMT: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.255 4/0 (hold time expired) 0 bytes
Sep  2 08:36:08.317 GMT: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 2): starting graceful recovery
Sep  2 08:36:08.317 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (2) is UP
Sep  2 08:36:08.801 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC UP, VC state UP
Sep  2 08:36:08.801 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC UP, VC state UP
Sep  2 08:36:08.801 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC UP, VC state UP
Sep  2 08:36:09.493 GMT: %OSPF-4-CONFLICTING_LSAID: LSA origination prevented by existing LSA with same LSID but a different mask
Sep  2 08:36:13.345 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.255 Up
Sep  2 08:36:57.138 GMT: %OSPF-4-CONFLICTING_LSAID: LSA origination prevented by existing LSA with same LSID but a different mask
Sep  2 08:37:07.955 GMT: %OSPF-4-CONFLICTING_LSAID: LSA origination prevented by existing LSA with same LSID but a different mask
Sep  2 08:38:08.325 GMT: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 2): completed graceful recovery
Sep  2 08:39:55.172 GMT: %LDP-5-GR: GR session 172.22.16.1:0 (inst. 2): interrupted--recovery pending
Sep  2 08:39:55.172 GMT: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.1:0 (0) is DOWN (Session KeepAlive Timer expired)
Sep  2 08:39:55.172 GMT: %XCONNECT-5-PW_STATUS: MPLS peer 172.22.16.1 vcid 200, VC DOWN, VC state DOWN
Sep  2 08:40:44.909 GMT: %BGP-5-ADJCHANGE: neighbor 172.22.16.255 Down BGP Notification sent
Sep  2 08:40:44.909 GMT: %BGP-3-NOTIFICATION: sent to neighbor 172.22.16.255 4/0 (hold time expired) 0 bytes
Sep  2 08:41:13.448 GMT: %OSPF-4-CONFLICTING_LSAID: LSA origination prevented by existing LSA with same LSID but a different mask


I'm problem only occurs when I bring up mpls between hong kong and London_Nr1.

I think it might be an issue with the route reflector but I'm not sure.

I have attached the configs and a topology.

Once again, I appreciate your help.

Regards

Carlton
hongkong.txt
London-Nr1.txt
London-Nr2.txt
route-reflector.txt
newshot.png
cpatte7372Asked:
Who is Participating?
 
Garry GlendownConsulting and Network/Security SpecialistCommented:
Oh, and strike my comment about OSPF earlier ... I was somewhat irritated by the network 172.22.16.x statement in the OSPF Router section ... As far as I know (and use) OSPF, the statements in all of the routers would just address the router themselves, and could therefore never set up a neighbor session with anybody else ... therefore, I believe they can and should be removed (though, if there is a reason for that line, please let me know ...)

While this also makes my idea of OSPF overwriting the static route wrong, at the moment I can only think of frame size limits causing the MPLS connection to fail ...
0
 
cpatte7372Author Commented:
Hello Experts,

I thought I would let you know that just before I was unable to connect to London_Nr1 I got the following messgae:

'The semaphore timeout period has expired'

This could be nothing but I got it on both occasions when I enabled mpls between Hong Kong and London_Nr1.

Cheers

Carlton
0
 
cpatte7372Author Commented:
Can I get some help with this please???
0
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

 
Garry GlendownConsulting and Network/Security SpecialistCommented:
If those configs are "complete", I'm missing configuration elements like MPLS MTU etc ...
Also, what frame size does your London1-HK link allow? Please do remember that MPLS will add between 4 (minimum) and 12 bytes (depending on the depth of label stacking - may even be higher in some special cases) to the frame size.

Try setting "ip mtu 1500" e.g. and see whether that fixes the problem. On some older systems, we also had to configure the "mpls mtu 1520".

Also with you using a VLAN, the problem may be caused by some switch on the way or the main interface's configuration ...
0
 
cpatte7372Author Commented:
Just so you after enabling the interface again between Hong Kong and London_Nr1 I experienced same problems see log  from Hong Kong below:

Sep  4 16:16:55: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 1): interrupted--recovery pending
Sep  4 16:16:55: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (0) is DOWN (Received error notification from peer: Holddown time expired)
Sep  4 16:19:44: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet1/0.998 from INIT to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:19:44: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet0/0.999 from INIT to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:35:13: %OSPF-4-NONEIGHBOR: Received database description from unknown neighbor 172.22.16.2
Sep  4 16:35:53: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet0/0.999 from EXSTART to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:35:53: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet1/0.998 from EXSTART to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:37:19: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (1) is UP
Sep  4 16:37:25: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet0/0.999 from LOADING to FULL, Loading Done
Sep  4 16:37:25: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet1/0.998 from LOADING to FULL, Loading Done
Sep  4 16:46:18: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 1): interrupted--recovery pending
Sep  4 16:46:18: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (0) is DOWN (Discovery Hello Hold Timer expired)
Sep  4 16:46:38: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet1/0.998 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:46:38: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet0/0.999 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep  4 16:46:49: %OSPF-5-ADJCHG: Process 100, Nbr 172.22.16.2 on GigabitEthernet1/0.998 from LOADING to FULL, Loading Done
Sep  4 16:46:50: %LDP-5-GR: GR session 172.22.16.2:0 (inst. 2): starting graceful recovery
Sep  4 16:46:50: %LDP-5-NBRCHG: LDP Neighbor 172.22.16.2:0 (2) is UP
0
 
cpatte7372Author Commented:
Do you know that the following means?

interrupted--recovery pending
0
 
Garry GlendownConsulting and Network/Security SpecialistCommented:
One (at least for me) weird thing I just noticed ... you're using a broadcast protocol (OSPF) to interconnect non-broadcast, /32 IP addresses? Didn't even know this could work ... why not do OSPF on the interconnection?

Anyway, one other possible cause of the failure could be that you static for the loopback addresses is overwritten by the dynamically learned address, causing both LDP and OSPF to fail afterwards ... give this a try: When OSPF is reporting "loading done", do a "show ip route 172.16.22.2" e.g. and see what you get ... I would bet you get a route that is causing the IP to be unreachable ...

If so, set up a filter in your OSPF processes to filter out the Loopback addresses from being distributed or imported ... (or use the 192.168. addresses for OSPF)
0
 
cpatte7372Author Commented:
Garry-G,

I think you hit the nail on the head. I also think OSPF is the cause. However, I can't test it right now as we can't afford to take our live network down.

Cheers

Carlton
0
 
cpatte7372Author Commented:
Garry-G

you're using a broadcast protocol (OSPF) to interconnect non-broadcast, /32 IP addresses

As I mentioned, I believe the above to be the cause, but I can't explain it. Can you please explain?

Cheers
0
 
Garry GlendownConsulting and Network/Security SpecialistCommented:
I've tried setting this up on GNS3 based on the limited configs you provided and it seems to work fine, both with and without MPLS on the LND-HK connection ... I can provide you with the GNS3 config files if you want to bring it up to your exact specs.

Anyway, one other thing (which may not apply to your setup) you should ensure you have looked into is the docs on inter-AS MPLS VPNs ...
0
 
cpatte7372Author Commented:
Thanks Garry.

Can I take a look at your GNS3 configs?

Cheers

Carlton
0
 
Garry GlendownConsulting and Network/Security SpecialistCommented:
will upload later ...
0
 
Garry GlendownConsulting and Network/Security SpecialistCommented:
Here's the config I put together ... Regretably, I don't know of any way to restrict technical settings of connections between locations (though, one could maybe also add a switch instead of a point-2-point connection, and configure the switch ports for limited MTU e.g.)
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.