Cisco Frame-relay trouble, not passing traffic, FR SVC disabled, LAPF state down

Posted on 2011-02-12
Medium Priority
Last Modified: 2012-05-11
Hello, as many time before ATT cannot fix a crucial frame-relay circuit. Usual the pvc is in a deleted state; however, this time it is active. The sho s0/1/0 shows an error of FR SVC disabled, LAPF state down

Any ideas what may be going wrong. Cisco sent a new csu/dsu, we even changed slots, and ATT claims to have completely rebuilt the circuit. Lastly, I sent them a loopback line from the router and the ATT tech said everything tested clean... Has anyone had any run ins on this one?

1009RTR01#                                     sho frame pvc

PVC Statistics for interface Serial0/1/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0


  input pkts 18            output pkts 1157         in bytes 1392      
  out bytes 106930         dropped pkts 0           in pkts dropped 0        
  out pkts dropped 0                out bytes dropped 0        
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0        
  out BECN pkts 0          in DE pkts 0             out DE pkts 0        
  out bcast pkts 1142      out bcast bytes 105970    
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 04:56:08, last time pvc status changed 00:19:58

1009RTR01#sho frame lmi
LMI Statistics for interface Serial0/1/0 (Frame Relay DTE) LMI TYPE = ANSI
  Invalid Unnumbered info 0             Invalid Prot Disc 0
  Invalid dummy Call Ref 0              Invalid Msg Type 0
  Invalid Status Message 0              Invalid Lock Shift 0
  Invalid Information ID 0              Invalid Report IE Len 0
  Invalid Report Request 0              Invalid Keep IE Len 0
  Num Status Enq. Sent 574              Num Status msgs Rcvd 545
  Num Update Status Rcvd 0              Num Status Timeouts 29
  Last Full Status Req 00:00:17         Last Full Status Rcvd 00:00:17

#sho int s0/1/0
Serial0/1/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: Connected to Cinemark_HQ
  MTU 1500 bytes, BW 384 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation FRAME-RELAY IETF, loopback not set
  Keepalive set (10 sec)
  LMI enq sent  576, LMI stat recvd 547, LMI upd recvd 0, DTE LMI up
  LMI enq recvd 25, LMI stat sent  0, LMI upd sent  0
  LMI DLCI 0  LMI type is ANSI Annex D  frame relay DTE
  FR SVC disabled, LAPF state down
  Broadcast queue 0/64, broadcasts sent/dropped 1150/0, interface broadcasts 1055
  Last input 00:00:04, output 00:00:00, output hang never
  Last clearing of "show interface" counters 04:59:09
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     2442 packets input, 161846 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 1 giants, 0 throttles
     377291 input errors, 377285 CRC, 163678 frame, 94780 overrun, 0 ignored, 284711 abort
     1738 packets output, 115512 bytes, 0 underruns
     0 output errors, 0 collisions, 118 interface resets
     0 output buffer failures, 0 output buffers swapped out
     9 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up
Question by:dcawood
  • 4
  • 4
LVL 13

Expert Comment

ID: 34880959
Every time I've seen this type of problem, It's been a telco problem.
Sounds like you need to escalate the problem.

Author Comment

ID: 34881230
Escalated it as high as I could. The switch manager at ATT admitted there was not a cabling issue but advised a vendor meet. Wow! After some arguing he finally agreed to have an engineer call me then I'll put another loopback on the router from the vsat backchannel. The level of knowledge is amazingly low. Even my bullmastiff agreed sending two cable techs to a site, causing three more days of outage time, for a site without layer 1 issues was nit the right choice. He was more vocal about it though.
LVL 13

Expert Comment

ID: 34881379
Well, your interface is showing up/up so I doubt any loopback test will tell you anything.
If you haven't already, I'd swap out the T1 card just to ensure there's nothing wrong on your end.
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.


Author Comment

ID: 34881419
Cisco sent a T1 card. Changed it out and moved it to a new slot for good measure. No Bueno. My bullmastiff was not impressed. ATT called a little while ago and said they might have a routing problem. Guess the have to bump into every wall before they find the door. I told them the problem straight away.
LVL 13

Accepted Solution

kdearing earned 2000 total points
ID: 34883318
Sometimes when I knew the problem was at their end, I used to trick them.

During troubleshooting, they would ask me to put a loopback on the circuit.
I'd say yes it's on and ask them if they could see it.
90% of the time they would say "yes we see the loopback"
That's when I would inform them I did not actually put the loopback on.
This usually caused great consternation and they would "discover" something on their end that caused the problem.

The sad thing is that it worked the vast majority of the time (dependent on the telco)

Author Comment

ID: 34883928
Yep. They refused to change ports initially. Then after they rebuilt the circuit who knows how many times and checked the routing from per to CER. They finally after hour and hours changed ports and the circuit came up. Every single time there is a problem like this a site takes a 5-10 day outage because they keep handing it back to the field and vendor meets. I end up spending hours and hours stopping this. The only recommendation that I can think of for my director and vp is that either our ATT service manager stops letting this happen on escalations or she be replaced. Hate to throw her under the bus but she's been acting as a glorified receptionist making sure the ticket moved in any direction instead of looking out for our best interest and making sure the right people kept the ticket till they fixed it. This outage was 5 days. The last outage was 10 days. Only good news is that all our circuits will be ppp by the summer.
LVL 13

Expert Comment

ID: 34884041
5 days?  10 days?  Wow
I think my longest was 3 days.
Need to beat them up over SLA, especially when it comes time for contract renewal.

I've got most of my clients on site-to-site vpn.
As long as internet is up, we control everything.

So, are you up and running now?

Author Comment

ID: 34943431
it was a bad port... ATT refused to change until they had to troubleshoot it for hours. Meanwhile I changed out all the remote equipment. The actually wanted to sent a truck out even though there was no Layer 1/2 issues.

Featured Post

Never miss a deadline with monday.com

The revolutionary project management tool is here!   Plan visually with a single glance and make sure your projects get done.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, we’ll look at how to deploy ProxySQL.
In this article, the configuration steps in Zabbix to monitor devices via SNMP will be discussed with some real examples on Cisco Router/Switch, Catalyst Switch, NAS Synology device.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

588 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question