Link to home
Start Free TrialLog in
Avatar of optimusone
optimusone

asked on

Cisco 1721 frame relay PVC not stable

What would make a frame relay PVC show as active for 10 seconds after router bootup or if the serial interface where forced administarively down then back up again. Internet traffic works for a few seconds then the PVC reports as deleted. I believe my hardware (cisco 1721 with T1DSU/CSU internal wic is not faulty. The provider is at a loss and wants to make internal changes on their side to different equipment.

Line protocol goes down. show frame pvc indicates no response from telco side. I am not onsite now so don't have access to the router.
Avatar of Jan Bacher
Jan Bacher
Flag of United States of America image

The keepalive default is 10 seconds -- that's what you're seeing.

Can you post the config snippet for the entire serial interface -- please leave the last two octets for the IPs as is and put 'X' for the first two if you would like to maintain privacy.
Avatar of optimusone
optimusone

ASKER

I can. I need to travel to the site to get on the console. I will get the config and post it.
Also grab a 'sho int' of the serial interface
will do. thought I should also provide you sh frame pvc and I bet it wouldnt hurt to debug frame lmi interface s0/1 as I did some indication from the ISP that they were seeing issue with LMI staying.

Outside of turing on screen capture on the terminal I use to connect to the console, how else can I capture the debug messages?
If you are doing logging, the debug messages print out  to the log (and the syslog server).  If the syslog server is remote, you are left with copy and paste.
Here is what you asked for plus a little more. I will post debug info next. Debug says LMI Type is invalid


Serial0 is up, line protocol is down                                    
  Hardware is PQUICC with Fractional T1 CSU/DSU                                              
  Description: connected to Internet                                    
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,                                
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation FRAME-RELAY IETF, loopback not set
  Keepalive set (10 sec)
  LMI enq sent  7613, LMI stat recvd 0, LMI upd recvd 0, DTE LMI down
  LMI enq recvd 0, LMI stat sent  0, LMI upd sent  0
  LMI DLCI 0  LMI type is ANSI Annex D  frame relay DTE
  Broadcast queue 0/64, broadcasts sent/dropped 0/0, interface broadcasts 0
  Last input 00:00:06, output 00:00:04, output hang never
  Last clearing of "show interface" counters 21:09:44
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     8585 packets input, 235465 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     2989 input errors, 624 CRC, 2294 frame, 0 overrun, 0 ignored, 70 abort
     7640 packets output, 108310 bytes, 0 underruns
     0 output errors, 0 collisions, 2537 interface resets
     0 output buffer failures, 0 output buffers swapped out
     5 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up


interface Serial0                
 description connected to Internet                                  
 no ip address              
 encapsulation frame-relay IETF                              
 no ip route-cache                  
 no fair-queue              
 service-module t1 timeslots 1-24                                
 service-module t1 remote-alarm-enable                                      
 service-module t1 fdl both                          
 cdp enable          
 frame-relay lmi-type ansi                          
!
interface Serial0.1 point-to-point                                  
 ip address x.x.140.186 255.255.255.252
 no ip route-cache
 no cdp enable
 frame-relay interface-dlci 645



PVC Statistics for interface Serial0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          0            0            1            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 645, DLCI USAGE = LOCAL, PVC STATUS = DELETED, INTERFACE = Serial0.1

  input pkts 1630          output pkts 27           in bytes 145102
  out bytes 1728           dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 0         out bcast bytes 0
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 21:12:55, last time pvc status changed 20:53:06
00:02:36: %LINK-5-CHANGED: Interface Serial0, changed state to administratively downno shut
EastGreenbush(config-if)#
00:02:49: %LINK-3-UPDOWN: Interface Serial0, changed state to down
00:03:19: %LINK-3-UPDOWN: Interface Serial0, changed state to up
00:03:27: Serial0: Invalid LMI type 1
00:03:29: Serial0(out): StEnq, myseq 1, yourseen 0, DTE up
00:03:29: datagramstart = 0x3507F94, datagramsize = 14
00:03:29: FR encap = 0x00010308
00:03:29: 00 75 95 01 01 00 03 02 01 00
00:03:29:
00:03:30: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0, changed state to up
00:03:38: Serial0: Invalid LMI type 1
00:03:39: Serial0(out): StEnq, myseq 2, yourseen 0, DTE up
00:03:39: datagramstart = 0x35085D4, datagramsize = 14
00:03:39: FR encap = 0x00010308
00:03:39: 00 75 95 01 01 00 03 02 02 00
00:03:39:
00:03:49: Serial0: Invalid LMI type 1
00:03:49: Serial0(out): StEnq, myseq 1, yourseen 0, DTE down
00:03:49: datagramstart = 0x3508AD4, datagramsize = 14
00:03:49: FR encap = 0x00010308
00:03:49: 00 75 95 01 01 00 03 02 01 00
Is this location going to be a host to other frame-relay remotes or is this just a remote?
just a remote. It is the Internet connection for a store. The ISP is delivering Internet via frame relay
I confirmed the frame relay specific config with the ISP tech yesterday. Invalid LMI type 1, from what I find means LMI mismatch. I thought you used either IETF or CISCO - understanding that the LMI type must match on the frame relay switch and the DTE.
this has been working for many many years and broke a few days ago. I just want to confirm that I dont have hardware problems. Looks to me that layer 1 is fine, layer 2 is the issue. What are the odds the T1 wic is bad and that is causing the LMI issue? I suspect the ISP but I am not 100%.
If it's a remote with no other upstreams, I don't bother with the subinterface.  It doesn't matter, just a personal preference.

config t
no interface Serial0.1 point-to-point

interface Serial0
 description connected to Internet
 ip address x.x.140.186 255.255.255.252
 no ip route-cache
 no ip redirects
 no ip proxy-arp
 no fair-queue
 no cdp enable
 service-module t1 timeslots 1-24
 service-module t1 clock source line
 service-module t1 remote-alarm-enable
 service-module t1 fdl both
 encapsulation frame-relay IETF
 frame-relay map ip x.x.140.185 645 broadcast
 frame-relay lmi-type ansi
 end

1) confirm full T1
2) confirm line coding is B8ZS
3) confirm framing is ESF
4) confirm frame-relay encapsulation is IETF
5) confirm lmi-type is ansi
6) presumes provider IP is x.x.140.185
7) usually frame networks provide clock source
8) do you need fdl  or can it be ignored?
1) confirm full T1 - CONFIRMED
2) confirm line coding is B8ZS- CONFIRMED
3) confirm framing is ESF- CONFIRMED
4) confirm frame-relay encapsulation is IETF- CONFIRMED
5) confirm lmi-type is ansi - CONFIRMED
6) presumes provider IP is x.x.140.185- CORRECT
7) usually frame networks provide clock source - HASNT been in the config but I can add it
8) do you need fdl  or can it be ignored? - I AM NOT SURE

What are your thoughts about why this broke. Again, the config with sub interface has been working for many years. Can DTE hardware flake out and keep LMI from syncing?
Your problem is that you're not getting any LMI status messages. There are a couple possible reasons:

1) You're using the wrong LMI type. Presently, you're hard coded to ANSI. It's possible the provider is using either Q933 or Cisco LMI's. Unless you're running an older IOS version, Cisco will autosense the LMI type. To enable it, remove the configuration parameter. "no frame-relay lmi-type ansi". Then let the router determine the correct LMI.

2) You could have a T1 problem. I don't think this is your issue since your interface is showing up.
Most locations I've done have been set to 'lmi-type ansi'.  Setup a frame circuit yesterday with a non-cisco remote and we had to change both ends to 'cisco' to bring line protocol up.

But you said that this identical configuration has worked for a long time?
to DONJOHNSTON - ISP tells me that LMI type must be ANSI

to JESPER- yes. this circuit has been up for 5+ years.

I want to do due diligence as the owner of the store doesnt care why it isnt working- he wants it fixed. I am not convinced it is hardware and dont want to buy new hardware that I will be stuck with if it is the ISP. The ISP told me that want to change hardware on their end on Monday and put the Internet connection through their ATM backbone and get it off their internal Frame Relay backbone. The last mile will continue to be Frame Relay, however.

I was hoping for some level of clarification of NO, this CANT be hardware on my side, or MOST LIKELY is not hardware on my side? Can either of you give me that peace of mind, or no?
And, yes I do have 5+ yr old IOS on the router.
If the IOS is 11.1 or earlier, then it will not auto-sense the LMI type. 11.2 or later will auto-sense.

Your debug frame lmi is just screaming that it's an LMI mismatch. If the interface were not up, you would not be sending LMI status inquiries. When you talk with your frame provider, are they saying that they are not receiving any LMI's? Are they seeing ANY traffic on the port your T1 terminates on?

The only other place to look is the T1 provider. Have them test the circuit and verify the end points. It's not unheard of for a T1 to get moved... by accident of course. ;-)
http://www.ccxx.net/books/Cisco.CCIE.Practical.Studies.Volume.I.pdf

says this:

An invalid LMI type of 1 indicates that the switch is
receiving Cisco LMI from the DTE end, thereby causing the timeout and the "down" condition.

That hurts my brain since there is a config line setting LMI to ANSI???????
Don- I struggle with an LMI mismatch if the provider has not changed anything at its end and this has been working and confirms it should be ansi.

I agree with opening up a trouble ticket on the circuit and having them run and end-to-end test.
The PVC comes up for 10 seconds or so during boot, or if I force the interface administratively down and bring it back up again. I can ping the internet during the time the PVC reports as active.

I heard the tech on the phone mumble something about LMI issues but hasnt said I have to make any changes to my config.

What is the possbility that the provider is upgrading their equipment and there is now a compatibility issue?
isnt there a service-module command I can add that will put my internal DSU/CSU into loopback. This way  the provider can do a full end to end test?
opps- I meant a command that puts the interface AUTOMATICALLY in loop then goes back to production when the provider is done looping?
They can put up a loop to you with no configuration at your end.   You don't configure a loop unless they specifically ask you to throw one on.
It is not unheard of for a provider to change LMI types. And yes, they also can "forget" to tell you.
looks like I will wait till Monday, cross my fingers this is a provider problem and get the line tested if there reprovisioning doesn't work.
SOLUTION
Avatar of Jan Bacher
Jan Bacher
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
>   2989 input errors, 624 CRC, 2294 frame, 0 overrun, 0 ignored, 70 abort
These error counters are from the telco side. I would open a trouble ticket with the telco and hold their feet to the fire to fix it.
Have you replaced the cable between the router and the T1 demarc?
If nothing changed on your end, the telco's upgrade and swap equipment all the time. I've had tech support swear up and down I need ANSI LMI, because that's what is in their records, not because that is what their equipment is set to. Take DonJohnsons advice and simply remove the lmi-type line from the config and that sets it to autosense.
added service-module t1 clock source line (default as line doesnt showup in config once it is added)
removed LMI ANSI line- version of IOS is 12.3
router auto? configured to LMI CISCO

SAME THINGS HAPPENS- PVC comes up after shut, not shut on int s0. Internet is pingable for a short period of time, then PVC deletes and connection goes down.

here are results of LMI debug with LMI type NOT forced to ANSI - no longer a mismatch reporting...but no response from teleco

1d00h: RT IE 1, length 1, type 0
1d00h: KA IE 3, length 2, yourseq 19, myseq 0
1d00h: Serial0(in): Unexpected StEnq
1d00h: Serial0(out): StEnq, myseq 3, yourseen 0, DTE down
1d00h: datagramstart = 0x3508354, datagramsize = 13
1d00h: FR encap = 0xFCF10309
1d00h: 00 75 01 01 00 03 02 03 00
1d00h:
1d00h: RT IE 1, length 1, type 0
1d00h: KA IE 3, length 2, yourseq 20, myseq 0
1d00h: Serial0(in): Unexpected StEnq
1d00h: Serial0(out): StEnq, myseq 4, yourseen 0, DTE down
1d00h: datagramstart = 0x3507314, datagramsize = 13
1d00h: FR encap = 0xFCF10309
1d00h: 00 75 01 01 00 03 02 04 00
1d00h:
1d00h: RT IE 1, length 1, type 0
1d00h: KA IE 3, length 2, yourseq 21, myseq 0
1d00h: Serial0(in): Unexpected StEnq
1d00h: Serial0(out): StEnq, myseq 5, yourseen 0, DTE down
1d00h: datagramstart = 0x3507954, datagramsize = 13
1d00h: FR encap = 0xFCF10309
1d00h: 00 75 01 01 00 03 02 05 00
Okay, a couple questions (and requests);

What IOS version are you running?
What LMI type are you seeing now? (show int)
When you say "Internet is pingable for a short period of time", what exactly do you mean? Specifically, what IP address are you successfully able to ping?

Finally, shut the interface down, clear the counters enable LMI debugging and bring the interface back up. Leave the debugging on for about two minutes and post the output of the debug and a "show interface".
I can ping 4.2.2.2 (vnsc-bak.sys.gtei.net) when the PVC becomes active at boot or after shut, no shut on s0
IOS is version 12.3
LMI is EITHER ANSI OR CISCO- When I set it auto by not having a config line in for lmi type, it comes up cisco. BOTH configs allow the connection to show PVC active for 10 seconds or so.

I havent been able to get back onsite to get to the router to get what you ask.

The provider is convinced this is my hardware problem and they are coming out with their own gear to prove it to the customer. They tell me that when they prove it, they will disconnect their CPE hardware and leave the circuit DOWN.

They wont do an end to end test of the line either- because the problem is "my hardware".
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Telco came onsite with their own demarc gear and RECREATED the problem. They then bypassed gear on their network and rebuilt the circuit to different gear.

Guess what???


THE CIRCUIT WORKS NOW!