Solved

PTP T-1 lines keep dropping

Posted on 2006-07-06
14
580 Views
Last Modified: 2012-08-13
I do not know what is going on but I cannot figure out why my T-1 lines keep going down. I've called sprint who is the provider. they are complaining that my HQ location is the problem. They can do a loopback test to the smartjack but after that they immediately get errors when looping from the smartjack to the CSU/DSU. I had a technician come in who is more familiar with T-1s than I am and he said my routing configuration was correct ESF B8ZS Clocking is provided by sprint.

both my lines typically go down at the same time. one line has in put and output errors all over the place on both sides. the other one does not but it still goes down. I've attached the out put of the show service serial x/x/x. any clue as to what I should check?? my cabling has tested good  and was verified by another technician.


Module type is T1/fractional                          
    Hardware revision is 1.2, Software revision is 20060106,                                                            
    Image checksum is 0x3EC4CA, Protocol revision is 0.1                                                        
Receiver has no alarms.                      
Framing is ESF, Line Code is B8ZS, Current clock source is line,                                                                
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.                                                                              
Last module self-test (done 13:52:09): Passed                                            
Last clearing of alarm counters 05:59:23                                        
    loss of signal        :    0,                                
    loss of frame         :    1, last occurred 03:22:43                                                        
    AIS alarm             :    1, last occurred 03:22:43                                                        
    Remote alarm          :    0,                                
    Module access errors  :    0,                                
Total Data (last 23 15 minute intervals):                                        
    0 Line Code Violations, 10769 Path Code Violations                                                      
    36 Slip Secs, 11311 Fr Loss Secs, 0 Line Err Secs, 156 Degraded Mins                                                                        
    1253 Errored Secs, 1227 Bursty Err Secs, 0 Severely Err Secs, 11311 Unavail                                                                              
Secs    
Data in current interval (845 seconds elapsed):                                              
    1 Line Code Violations, 443 Path Code Violations                                                    
    1 Slip Secs, 480 Fr Loss Secs, 1 Line Err Secs, 6 Degraded Mins            
    46 Errored Secs, 45 Bursty Err Secs, 0 Severely Err Secs, 480 Unavail Secs
LEO_2821#show service serial 0/1/0
Module type is T1/fractional
    Hardware revision is 1.2, Software revision is 20060106,
    Image checksum is 0x3EC4CA, Protocol revision is 0.1
Receiver has no alarms.
Framing is ESF, Line Code is B8ZS, Current clock source is line,
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.
Last module self-test (done 13:51:19): Passed
Last clearing of alarm counters 06:12:22
    loss of signal        :    0,
    loss of frame         :    1, last occurred 03:22:49
    AIS alarm             :    1, last occurred 03:22:49
    Remote alarm          :    0,
    Module access errors  :    0,
Total Data (last 24 15 minute intervals):
    1 Line Code Violations, 1 Path Code Violations
    0 Slip Secs, 11437 Fr Loss Secs, 1 Line Err Secs, 0 Degraded Mins
    2 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 11437 Unavail Secs
Data in current interval (722 seconds elapsed):
    1 Line Code Violations, 1 Path Code Violations
    0 Slip Secs, 353 Fr Loss Secs, 1 Line Err Secs, 2 Degraded Mins
    1 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 353 Unavail Secs
0
Comment
Question by:jbla9028
  • 6
  • 5
  • 2
  • +1
14 Comments
 
LVL 32

Expert Comment

by:rsivanandan
ID: 17056255
1. Cabling is checked
2. Correct Cable
3. ISP says they don't have a problem (Well, have to believe them :-))

What kind of routers do you have on both ends ? Have tried clearing the counters on the interface on both routers and keep watching how the input/output errors are accumulating ?

Cheers,
Rajesh
0
 
LVL 44

Expert Comment

by:scrathcyboy
ID: 17056290
No, never believe an ISP or a Telco provider, they always pass the buck to someone else.  Are these CSU/DSU units leased or bought?  If leased, tell sprint to replace them, that will get there attention real fast, that this is going to cost them money.

"They can do a loopback test to the smartjack but after that they immediately get errors when looping from the smartjack to the CSU/DSU"

This is definitiely a hardware/communication problem on their side of the line, I would work on them.
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 17057569
Agree with Raj - check the cabling between the smartjack and your DSU.
This appears to be output from a Cisco 1800/2800 series router?
Are these point-to-point T1's? If yes, are you providing clocking at one end? Which end?

interface serial0/0/0
 service-module t1 clock source internal

Do that on one end only and the other end will syncronize the T1 clock timeing with it.

What encapsulation are you using on these T1's ? HDLC?
0
 
LVL 1

Author Comment

by:jbla9028
ID: 17057828
Thanks for the conmments.

encapsulation is HDLC
I have premade straight thru cables going from the smart jack to the DSU/CSU card.
my routers are the following
HQ in MA  Cisco 2821 w/ two internal DSU/CSU cards

T-1 #1 MA-AZ
Cisco 1841 w/ 1 internal DSU/CSU card

T-1 #2 MA-MS
Cisco 1841 w 1 internal DSU/CSU card

All equipment is brand new. I've called sprint and they have assured me that all clocking is being provided by them. If I put clocking on the lines they go down and never come back up. whenever I find that the connection is going back up I do a clear counters serial x/x/x. on the AZ T1 I see errors immediately here is what I walked into work showing
This is the AZ T-1
-----------------------------------------------------------------------------
LEO_2821#show int s0/0/0
Serial0/0/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Internet address is 10.200.200.189/30
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 212/255, txload 1/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:01, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/1/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1158 kilobits/sec
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     9774 packets input, 651042 bytes, 0 no buffer
     Received 2962 broadcasts, 0 runts, 1135 giants, 0 throttles
     485190 input errors, 485189 CRC, 280119 frame, 47990 overrun, 0 ignored, 14
5633 abort
     8542 packets output, 558939 bytes, 0 underruns
     0 output errors, 0 collisions, 4 interface resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up
------------------------------------------------------------------------------
This is the MS T-1
-----------------------------------------------------------------------------
LEO_2821#show int s0/1/0
Serial0/1/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Internet address is 10.200.200.181/30
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:01, output 00:00:04, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/1/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1152 kilobits/sec
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     8523 packets input, 558492 bytes, 0 no buffer
     Received 2986 broadcasts, 0 runts, 1 giants, 0 throttles
     11722 input errors, 11721 CRC, 4957 frame, 1083 overrun, 0 ignored, 6622 abort
     8526 packets output, 560084 bytes, 0 underruns
     0 output errors, 0 collisions, 6 interface resets
     0 output buffer failures, 0 output buffers swapped out
     7 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up
------------------------------------------------------------------------
Sprint came in and said nothing was wrong but for some reason verizon came and did some work on the lines... why would they do work if nothing is wrong....

next course is I have a technician from a local company bringing in some external CSUs. we're going to hook those up and then have sprint test from the smart jack to the CSUs.

I'm convinced that this is a provider problem. why would both lines drop at the same time?? the only other explaination is that my router is bad. I am going to pick up a spare 1600 series router off ebay if the CSU issue doesn't give us any insite.


0
 
LVL 79

Expert Comment

by:lrmoore
ID: 17057935
> I've called sprint and they have assured me that all clocking is being provided by them.
Makes sense on such long distance lines..

Tell Sprint to quit dinking around and change these P-2-P T1's to MPLS
You can have 2x T1 into the MPLS "cloud" and each remote has a T1 into the same cloud. The cloud does the routing magic using BGP4.
>  11722 input errors, 11721 CRC,
These are almost always carrier issues or wiring issues. Are your "premade" cables standard CAT5 patch cables? I've found these to work the best.
Have you run a "hard loop" to have Sprint try to loop to past the smartjack? Pull the cable end out of your router and plug in a hard loopback plug and see if they can see it then.
http://www.smarter.com/sa_product.php?prodid=454472870&q=t1+loopback+plug
0
 
LVL 1

Author Comment

by:jbla9028
ID: 17058026
I will purchase the loop back plugs. I will change out my cabling again here jsut the make sure. if my cabling was messed up wouldn't the technician not be able to loop to the smart jack clean?  

what is an input error? I take it the CRC error is saying that of the packets received X packets failed due to not passing a checksum test.

The sprint tech is on the phone and he is claiming that he can loop back and forth between the AZ-MA CSUs fine. am I now clean on the physical side and now should be concerned with my datalink layer and network layer side?
0
 
LVL 79

Accepted Solution

by:
lrmoore earned 500 total points
ID: 17058131
Input errors on your CSU means from the ISP to you.
Good troubleshooting doc to keep handy:
http://www.cisco.com/en/US/tech/tk713/tk628/technologies_tech_note09186a00800a758d.shtml

datalink layer = HDLC - I don't think that's the problem
Network layer = IP addressing and routing. This does not cause CRC errrors - ever.

If they have apparently cleared up the issues on their end (they will never admit to having anything to do with it), I'd clear the counters and not allow them to close the ticket until you run error-free for 72 hours. They won't like it but you're the customer and can insist on it.


0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 1

Author Comment

by:jbla9028
ID: 17058162
the sprint guy just went through every single one of the legs of the T-1 on the AZ-MA T1 and found that for some reason they were not providing Timing between one of the legs after he changed that I got no input or output errors on that T-1. I've pinged with all 000s and all 111s and I'm clean so far. he's checking the MS-MA t1 right now.
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 17058216
Sounds like progress!!
Keep you fingers crossed...
0
 
LVL 1

Author Comment

by:jbla9028
ID: 17058281
i just don't understand why they can't see this crap on there end I swear this is like the 8th time I've called them. This costs my company so much money and grief they should give me the T-1 for Free!!!
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 17058376
Quite typical of all the telcos. You really have to assert yourself and request escalation if you are not satisfied the first time you call. Usually the 2nd or 3rd time usually you'll get someone that actually knows what they're doing and sees the problem, fixes it right away..
0
 
LVL 44

Expert Comment

by:scrathcyboy
ID: 17061148
" i just don't understand why they can't see this crap on there end I swear this is like the 8th time I've called them."

They CAN.  They just dont want to put in the effort, they want the user to do it.  Sorry, but this is typical of sprint -- atrocious customer support, always has been.  

Too bad you cant go to 5 Mbit DSL -- it will run rings around yor current setup.
And 10Mbit DSL is right around the corner.  That is modern progress, T1 is not.
0
 
LVL 1

Author Comment

by:jbla9028
ID: 17068812
welll I'm still having issues with my T-1 provider I think. I'm looking at the router and still seeing tons of input and CRC errors from both lines. I am going to pick up some external CSUs to see if we can eliminate the problems that we are having.
0
 
LVL 1

Author Comment

by:jbla9028
ID: 17096597
I actually figured out what the issue was last night That I have been having with dropping. The timing that the provider was giving me was causing some serious issues. I had the provider disable their timing and I used my own on my masschusetts router and all is well!
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

#Citrix #Citrix Netscaler #HTTP Compression #Load Balance
Don’t let your business fall victim to the coming apocalypse – use our Survival Guide for the Fax Apocalypse to identify the risks and signs of zombie fax activities at your business.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now