jschillinger2
asked on
T1 errors, drops - Cisco
Ok, this is a really strange one... I have a Location A which has three T1s from the same telco (Windstream, formerly Alltel). Two are point to point and one is a PRI. One of the point to point T1s is taking a lot of errors, inconsistently, and dropping. It comes right back up and the cycle continues. The telco is providing clock from the CO to both ends of this circuit. So, both ends are configured to take clock from the line. The circuit will run clean for hours or days, then the error rate goes through the roof and it it'll bounce for a while. The telco has been out to test both ends several times, as well as testing intrusively twice for two hours. No errors from smartjack to smartjack. Last week I called and the tech was out again monitoring at the smartjack at Location A with a tbird. He was seeing errors coming from the router on the Site A side, and none from Site B. While we were talking, we both saw the linecode indicator on the tbird flash back and forth between AMI and B8ZS. It changed pretty rapidly for a few seconds, then settled back to B8ZS which it is supposed to be. How is this possible, and what do I do about it?
Location A Equipment:
Cisco 2821 ISR
VWIC-2MFT-T1 (PRI and the other point-to-point)
WIC-1T (the T1 in question)
KENTROX CSU/DSU, v.35 cable
Location B Equipment:
Cisco 2811 ISR
VWIC-1MFT-T1 (the T1 in question)
Here's what I've done so far:
1. Changed equipment. I've tried this T1 on both the WIC controller and the external DSU.
2. Changed clock source. I've tried line/line, line/internal, internal/line. Haven't tried internal/internal 'cuz you know that wouldn't work...
3. Ran a temporary cable from the DSU to the smartjack to make sure it wasn't the building wiring. No change.
I'm really out of ideas. I've changed everything at Location A, and everything but the wiring at Location B. The telco insists the circuit is clean and that it is my equipment. After seeing those results on the tbird last week, I'm convinced, but HOW IS THIS POSSIBLE?? The other two T1s run completely clean at that location, on the same equipment that has been tried on the problem T1. I don't use line build out settings on the other T1s. All are ESF/B8ZS. All the equipment has been working in that location or other locations. We run VoIP over this link and it causes quite the headache for Location B when they have no phone service when the circuit drops.
Any help is very greatly, truly appreciated!
Location A Equipment:
Cisco 2821 ISR
VWIC-2MFT-T1 (PRI and the other point-to-point)
WIC-1T (the T1 in question)
KENTROX CSU/DSU, v.35 cable
Location B Equipment:
Cisco 2811 ISR
VWIC-1MFT-T1 (the T1 in question)
Here's what I've done so far:
1. Changed equipment. I've tried this T1 on both the WIC controller and the external DSU.
2. Changed clock source. I've tried line/line, line/internal, internal/line. Haven't tried internal/internal 'cuz you know that wouldn't work...
3. Ran a temporary cable from the DSU to the smartjack to make sure it wasn't the building wiring. No change.
I'm really out of ideas. I've changed everything at Location A, and everything but the wiring at Location B. The telco insists the circuit is clean and that it is my equipment. After seeing those results on the tbird last week, I'm convinced, but HOW IS THIS POSSIBLE?? The other two T1s run completely clean at that location, on the same equipment that has been tried on the problem T1. I don't use line build out settings on the other T1s. All are ESF/B8ZS. All the equipment has been working in that location or other locations. We run VoIP over this link and it causes quite the headache for Location B when they have no phone service when the circuit drops.
Any help is very greatly, truly appreciated!
I have had similar cases with my T1's (I have 56). In one case it was a cross connect jumper in the CO. I have also had a bad socket in a smart jack cabinet. In that case I moved all of the wiring myself over to a blank space in the smart jack cabinet. It fixed the problem. I would escalate it with an engineer.
ASKER
Ok guys, I'll try to get the smartjack moved. I'll get back to you, thanks!!!
ASKER
Ok, that wasn't it. The smartjack is moved to the slot where my PRI was and still getting massive errors and drops. The smartjack doesn't see any errors, nor the CO. Almost all the errors are on the Location A side. The error rate on Location B is less than 1% of what A sees. Any other ideas? Thanks in advance!
OK... you stated in your problem that you have:
Location A Equipment:
Cisco 2821 ISR
VWIC-2MFT-T1 (PRI and the other point-to-point)
WIC-1T (the T1 in question)
KENTROX CSU/DSU, v.35 cable
Location B Equipment:
Cisco 2811 ISR
VWIC-1MFT-T1 (the T1 in question)
Are you connecting a DS1 (T1) to two different types of interfaces on each end? If so this would produce line coding errors.
Location A Equipment:
Cisco 2821 ISR
VWIC-2MFT-T1 (PRI and the other point-to-point)
WIC-1T (the T1 in question)
KENTROX CSU/DSU, v.35 cable
Location B Equipment:
Cisco 2811 ISR
VWIC-1MFT-T1 (the T1 in question)
Are you connecting a DS1 (T1) to two different types of interfaces on each end? If so this would produce line coding errors.
ASKER
How so? The WIC-1T is a serial interface card. That is attached to a DSU. On the other end is the VWIC-1MFT-T1 which is a T1 controller card (DSU+serial interface). The combination of the serial card + DSU is equivalent to the VWIC-1MFT-T1. And in any case, the original configuration had the VWIC-1MFT-T1 and the VWIC-2MFT-T1 paired with the same result. One of my other T1s has different WICs on each end and works fine.
So you have the Kentrox CSU/DSU connected to the WIC-T1? Just so I am clear on what card it is can you give me the part number of the WIC-T1? It also may be that the Kentrox is faulty or not configured correctly.
ASKER
WIC-1T is the Cisco model number. I have swapped the Kentrox DSU with a known good one. I have double checked the configuration on the DSU, but it hasn't changed. Setting are the same on both point-to-points at that location and I can swap them back and forth between the WIC-1T + Kentrox DSU and the VWIC-2MFT-T1 and the errors always stay on the problem T1.
Another update is that the circuit has been running clean since 6:10pm on Friday... Makes me wonder if there is a source of interference that occurs during normal business hours M-F. I can't recall right now if the problems occur over the weekend or not. Again though, all three T1s at that location follow the exact same path according to the telco tech. And all three follow the same path in the building. If that were the case, all three should have the same problem...
I replaced the UPS powering the router and DSU yesterday in case some weird power glitch was causing problems on just that one T1 but no change. Still observed the errors and circuit dropped after changing the UPS.
Hopefully on Monday, the tech is going to switch the circuit legs from the remote office to my site of the PRI and the problem T1. The PRI is known good, so we'll see if that makes any difference.
Thanks!!
Another update is that the circuit has been running clean since 6:10pm on Friday... Makes me wonder if there is a source of interference that occurs during normal business hours M-F. I can't recall right now if the problems occur over the weekend or not. Again though, all three T1s at that location follow the exact same path according to the telco tech. And all three follow the same path in the building. If that were the case, all three should have the same problem...
I replaced the UPS powering the router and DSU yesterday in case some weird power glitch was causing problems on just that one T1 but no change. Still observed the errors and circuit dropped after changing the UPS.
Hopefully on Monday, the tech is going to switch the circuit legs from the remote office to my site of the PRI and the problem T1. The PRI is known good, so we'll see if that makes any difference.
Thanks!!
ASKER
OK... I see now. I was confused for a minute. I haven't seen one on those in several years. The thing I keep going back to is when you mentioned that when the telco tech was monitoring your smart jack and observed it changing rapidly from B8ZS to AMI. The T-bird was getting that signal from somewhere on the CPE side of the smart jack. Was your equipment plugged in at the time or was the smart jack looped up?
ASKER
The equipment was plugged in and the circuit was up, then it was taking errors while we saw the linecode changing and it bounced a few times. It all seems to indicate a problem on our side, but I can't come up with an explanation or a fix. I have yet another hardware swap to try, a known-good WIC-1DSU-T1-V2. At this point the circuit has still been clean since Friday evening, so I'm not sure I want to change anything now. I'll check it tomorrow.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I practicially had to drive down to AT&T and beat them into submission. You just have to keep at it and hope you get a good tech who will go the extra mile. Buy him lunch, whatever.