asked on

50% packet loss

I just setup a new Bundled T1 Internet connection through AT&T on a Cisco 2901 router. From the router I can ping public IPs fine. From my laptop on the LAN side of the network I get exactly a 50% packet loss pinging the same address. However, I can tie my laptop back into the old ISP connection and ping the same IP address just fine. Any thoughts? Here's my router config:

HDORouter#show config
Using 1938 out of 262136 bytes
!
! Last configuration change at 21:30:06 UTC Tue Sep 21 2010
!
version 15.0
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname HDORouter
!
boot-start-marker
boot-end-marker
!
card type t1 0 0
card type t1 0 1

!
no aaa new-model
no network-clock-participate wic 0
no network-clock-participate wic 1
!
no ipv6 cef
ip source-route
ip cef
!
!
!
!
ip domain name usavgroup.local
multilink bundle-name authenticated
!
!
!
!
license udi pid CISCO2901/K9 sn FTX141181FY
!
!

!
!
controller T1 0/0/0
cablelength long 0db
!
controller T1 0/0/1
cablelength long 0db
!
controller T1 0/1/0
cablelength long 0db
channel-group 0 timeslots 1-24
!
controller T1 0/1/1
cablelength long 0db
channel-group 0 timeslots 1-24
!
!
!
!
!
interface Multilink1
description connected to ISP
ip address 12.90.105.14 255.255.255.252
ppp multilink
ppp multilink group 1
ppp multilink fragment disable
!
interface GigabitEthernet0/0
description $ES_LAN$
ip address 12.175.223.225 255.255.255.248
duplex auto
speed auto
!
interface GigabitEthernet0/1
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/1/0:0
description multilink 1 interface
no ip address
encapsulation ppp
tx-ring-limit 26
tx-queue-limit 26
no keepalive
ppp multilink
ppp multilink group 1
no fair-queue
!
interface Serial0/1/1:0
description multilink 1 interface
no ip address
encapsulation ppp
tx-ring-limit 26
tx-queue-limit 26
ppp multilink
ppp multilink group 1
!
ip forward-protocol nd
!
ip http server
no ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 12.90.105.13
!
!
!
snmp-server community xxxxxxx RW
!
control-plane
!
!
line con 0
line aux 0
line vty 0 4
password xxxxxxxxxx
login
!
scheduler allocate 20000 1000
end

HDORouter#

ASKER CERTIFIED SOLUTION

rfc1180

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

MasterComputing

ASKER

HDORouter>show int multilink1
Multilink1 is up, line protocol is up
Hardware is multilink group interface
Description: connected to ISP
Internet address is 12.90.105.14/30
MTU 1500 bytes, BW 3072 Kbit/sec, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
REQsent: CDPCP
Open: IPCP, loopback not set
Keepalive set (10 sec)
DTR is pulsed for 2 seconds on reset
Last input 00:00:00, output never, output hang never
Last clearing of "show interface" counters 00:32:56
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1000 bits/sec, 1 packets/sec
5 minute output rate 1000 bits/sec, 1 packets/sec
1045 packets input, 70345 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
2583 packets output, 167348 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
HDORouter>show int gigabitethernet 0/0
GigabitEthernet0/0 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is 68ef.bd86.7ab8 (bia 68ef.bd86.7ab8
)
Description: $ES_LAN$
Internet address is 12.175.223.225/29
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is RJ45
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:03:29, output 00:00:12, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
1552 packets input, 123186 bytes, 0 no buffer
Received 30 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
625 packets output, 59603 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
1 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

HDORouter>show controllers gigabitethernet 0/0
Interface GigabitEthernet0/0
= Hardware =
Hardware is PLATFORM SGMII INTERFACE
HWIDB: 2929B484, INSTANCE: 3084D400, FASTSEND: 211883E8, MCI_INDEX: 0
Route Cache Flag: 11

= Media RJ45 =

Media Change Count: 1

PORT IPD Registers:
BP_PAGE_CNT: 0x0000000000020001, BP_CNTS_PRX: 0x0000000000000000
GMX Registers:
RXX_INT_REG: 0x0000000000000000, RXX_INT_EN : 0x0000000000000000
PRTX_CFG : 0x000000000000300F, RXX_FRM_CTL: 0x000000000000007F
RXX_FRM_CHK: 0x00000000000003BA, RXX_FRM_MIN: 0x0000000000000000
RXX_FRM_MAX: 0x0000000000000000, RXX_JABBER : 0x0000000000002580
RXX_DECISIO: 0x0000000000000018, RXX_FRM_CTL: 0x000000000000007F
RXX_FRM_CHK: 0x00000000000003BA, RXX_JABBER : 0x0000000000002580
RXX_DECISIO: 0x0000000000000018, RXX_UDD_SKI: 0x0000000000000000
RXX_STATS_C: 0x0000000000000000, RXX_IFG : 0x0000000000000008
RXX_PAU_DRP: 0x0000000000000000, RXX_STATS_P: 0x0000000000000935
RXX_STAT_CT: 0x0000000000000000, RXX_STATS_D: 0x0000000000000000
RXX_STAT_DR: 0x0000000000000000, RXX_STATS_B: 0x0000000000000000
SMACX : 0x000068EFBD867AB8
TXX_SGMII_C: 0x0000000000000001, TXX_CTL : 0x0000000000000003
RX_PRTS : 0x0000000000000004, RX_BP_DROP : 0x0000000000000002
RX_BP_ON : 0x0000000000000040, RX_BP_OFF : 0x0000000000000010
TXX_PAU_PKT: 0x0000000000000060, TXX_MIN_PKT: 0x000000000000003B
TXX_PAU_PKI: 0x0000000000000010, TXX_SOFTPAU: 0x0000000000000000
TXX_PAU_TGO: 0x0000000000000000, TXX_PAU_ZRO: 0x0000000000000001
INF_MODE : 0x0000000000000022, TX_PRTS : 0x0000000000000004
TX_IFG : 0x0000000000000048, TX_JAM : 0x00000000000000EE
TX_COL_ATMP: 0x0000000000000010, TX_PAU_DMAC: 0x00000180C2000001
TX_PAU_TYPE: 0x0000000000008808, TX_OVR_BP : 0x0000000000000A0A
TX_BP : 0x0000000000000000, TX_CORRUPT : 0x000000000000000F
RX_PRT_INFO: 0x0000000000000000, GMX_STAT_BP: 0x0000000000000000
TX_INT_REG : 0x0000000000000000, TX_INT_EN : 0x0000000000000000
ADR_CAM0 : 0x0000000000000068
ADR_CAM1 : 0x00000000000000EF
ADR_CAM2 : 0x00000000000000BD
ADR_CAM3 : 0x0000000000000086
ADR_CAM4 : 0x000000000000007A
ADR_CAM5 : 0x00000000000000B8
ADR_CTL : 0x000000000000000D, CAM_EN : 0x00000000000000FF
TXX_STAT0 : 0x0000000000000000, TXX_STAT1 : 0x0000000000000000
TXX_STAT2 : 0x000000000000F5E4, TXX_STAT3 : 0x000000000000027C
TXX_STAT4 : 0x000000E700000000, TXX_STAT5 : 0x0000000C00000160
TXX_STAT6 : 0x0000000100000027, TXX_STAT7 : 0x0000000000000001
TXX_STAT8 : 0x0000002A0000001B, TXX_STAT9 : 0x0000000000000000
PCS Registers:
MRX_CTL_REG: 0x0000000000001140, MRX_STS_REG: 0x000000000000017C
RXX_STA_REG: 0x00000000000008C7, RXX_SYN_REG: 0x0000000000000003
TXX_STA_REG: 0x0000000000000064, SGMX_AN_ADV: 0x0000000000001801
SGMX_AN_ADV: 0x000000000000D801, MISC_CTL : 0x0000000000000000
ANX_ADV_REG: 0x0000000000000060, ANX_LP_ABIL: 0x000000000000D000
ANX_RESULTS: 0x0000000000000017, ANX_EXT_ST : 0x000000000000C000
LNX_TMR_CNT: 0x000000000000034A, TX_RXX_PLRT: 0x0000000000000000
INT_REG : 0x0000000000000000, INT_EN_REG : 0x0000000000000000
LOG_ANL_REG: 0x0000000000000000, ANX_ADV_REG: 0x0000000000000060
ANX_LP_ABIL: 0x000000000000D000, ANX_RSL_REG: 0x0000000000000017
ANX_EXT_ST : 0x000000000000C000, LNX_TMR_CNT: 0x000000000000034A
MISCX_CTL: 0x0000000000001001
==============PIP Statistics================

PIP_RX_NO_ERR = 0, PIP_PARTIAL_ERR = 0
PIP_JABBER_ERR = 0, PIP_OVER_FCS_ERR = 0
PIP_OVER_ERR = 0, PIP_ALIGN_ERR = 0
PIP_UNDER_FCS_ERR = 0, PIP_GMX_FCS_ERR = 0
PIP_UNDER_ERR = 0, PIP_EXTEND_ERR = 0
PIP_LENGTH_ERR = 0, PIP_DAT_ERR = 0
PIP_SKIP_ERR = 0, PIP_NIBBLE_ERR = 0
PIP_UNKNOWN_ERR_14 = 0, PIP_UNKNOWN_ERR_15 = 0
PIP_PIP_FCS = 0, PIP_PIP_SKIP_ERR = 0
PIP_PIP_L2_MAL_HDR = 0
pip_rx_no_particle = 0, pip_undef_err = 0
pip_rx_no_buf_start = 0

PHY REGISTERS:
page number:0

Copper Contrl @ reg_addr: 0: 0x1140
Copper Status @ reg_addr: 1: 0x796D
PHY Idf1 @ reg_addr: 2: 0x0141
PHY Idf2 @ reg_addr: 3: 0x0C97
Autoneg Adv @ reg_addr: 4: 0x0DE1
Autoneg Lnk Ptr abty @ reg_addr: 5: 0xC5E1
Autoneg Exp @ reg_addr: 6: 0x000D
Autoneg Nx Pg Tx @ reg_addr: 7: 0x2801
Autoneg Lk Prt Nx Pg @ reg_addr: 8: 0x4FBF
1000B-T Ctr @ reg_addr: 9: 0x0E00
1000B-T St @ reg_addr: 10: 0x7C00
Copper Ext St @ reg_addr: 15: 0x3000
Copper Specific Ctl1 @ reg_addr: 16: 0x6060
Copper Specific St1 @ reg_addr: 17: 0xAF08
Copper Interrupt En @ reg_addr: 18: 0x0000
Copper Specific St2 @ reg_addr: 19: 0x0000
Copper Rcv Er Ctr @ reg_addr: 21: 0x0000
Copper Specific Ctl2 @ reg_addr: 26: 0x8040

page number:1

Fiber Contrl @ reg_addr: 0: 0x1140
Fiber Status @ reg_addr: 1: 0x794D
PHY Idf1 @ reg_addr: 2: 0x0141
PHY Idf2 @ reg_addr: 3: 0x0C97
Autoneg Adv @ reg_addr: 4: 0x0060
Autoneg Lnk Ptr abty @ reg_addr: 5: 0x0000
Autoneg Exp @ reg_addr: 6: 0x0004
Autoneg Nx Pg Tx @ reg_addr: 7: 0x2001
Autoneg Lk Prt Nx Pg @ reg_addr: 8: 0x0000
Fiber Ext St @ reg_addr: 15: 0x3000
Fiber Specific Ctl1 @ reg_addr: 16: 0x0000
Fiber Specific St1 @ reg_addr: 17: 0x0018
Fiber Interrupt En @ reg_addr: 18: 0x0000
Fiber Specific St2 @ reg_addr: 19: 0x0000
Fiber Rcv Er Ctr @ reg_addr: 21: 0x0000
Fiber Specific Ctl2 @ reg_addr: 26: 0x8085

page number:2

MAC Contrl @ reg_addr: 0: 0x1040
PHY Idf1 @ reg_addr: 2: 0x0141
PHY Idf2 @ reg_addr: 3: 0x0C97
MAC Specific Ctl1 @ reg_addr: 16: 0x0288
MAC Specific St1 @ reg_addr: 17: 0x0440
MAC Interrupt En @ reg_addr: 18: 0x0000
MAC Specific St2 @ reg_addr: 19: 0x0000
MAC Specific Ctl2 @ reg_addr: 26: 0x0005

TX_COUNT: 0, TX_LIMITED: 0, TX_HEAD: 122, TX_TAIL: 122
TX_MIN_SIZE: 32, TX_MAX_SIZE: 512, TX_CURRENT_SIZE: 128
RX STARVES: 0

Software MAC Address Filter (hash:length/addr/mask/hits)
--------------------------------------------------------
0x000: 0 ffff.ffff.ffff 0000.0000.0000 48
0x005: 0 68ef.bd86.7ab8 0000.0000.0000 0
0x0C0: 0 0180.c200.0002 0000.0000.0000 0
0x0C0: 1 0100.0ccc.cccc 0000.0000.0000 0
0x0C5: 0 0180.c200.0007 0000.0000.0000 0

Software filtered frames: 782
Unicast overflow mode: 0
Multicast overflow mode: 1
Promiscuous mode: 0
Total Number of CAM entries: 1
Port Stopped: N

Internal Loopback Set: N

HDORouter>show int serial 0/1/0:0
Serial0/1/0:0 is up, line protocol is up
Hardware is DSX1
Description: multilink 1 interface
MTU 1500 bytes, BW 1536 Kbit/sec, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Multilink1, loopback not set
Keepalive not set
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 00:34:07
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 1 packets/sec
5 minute output rate 0 bits/sec, 1 packets/sec
838 packets input, 46976 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
1617 packets output, 98975 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions
Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags
HDORouter>show int serial 0/1/1:0
Serial0/1/1:0 is up, line protocol is up
Hardware is DSX1
Description: multilink 1 interface
MTU 1500 bytes, BW 1536 Kbit/sec, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Multilink1, loopback not set
Keepalive set (10 sec)
Last input 00:00:02, output 00:00:00, output hang never
Last clearing of "show interface" counters 00:34:18
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair [suspended, using FIFO]
FIFO output queue 0/40, 0 drops
5 minute input rate 0 bits/sec, 1 packets/sec
5 minute output rate 0 bits/sec, 1 packets/sec
1050 packets input, 50679 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
1834 packets output, 101165 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions
Timeslot(s) Used:1-24, SCC: 1, Transmitter delay is 0 flags
HDORouter>

rfc1180

try
conf t

no ip route 0.0.0.0 0.0.0.0 12.90.105.13
ip route 0.0.0.0 0.0.0.0 Multilink1
exit
wr mem

rfc1180

if you are remote:

try
conf t

ip route 0.0.0.0 0.0.0.0 Multilink1
no ip route 0.0.0.0 0.0.0.0 12.90.105.13
exit
wr mem

MasterComputing

ASKER

Here's the new routing table. Still no luck.

HDORouter#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, + - replicated route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

S* 0.0.0.0/0 is directly connected, Multilink1
12.0.0.0/8 is variably subnetted, 5 subnets, 3 masks
C 12.90.105.12/30 is directly connected, Multilink1
C 12.90.105.13/32 is directly connected, Multilink1
L 12.90.105.14/32 is directly connected, Multilink1
C 12.175.223.224/29 is directly connected, GigabitEthernet0/0
L 12.175.223.225/32 is directly connected, GigabitEthernet0/0
HDORouter#

rfc1180

please post:

I am thinking something is up with CEF, though for sure that updating the route will fix the issue. Pinging the multilink interface does not give any packet loss.

show ppp multilink
show ip cef
show ip int

Billy

rfc1180

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair [suspended, using FIFO]

I believe that might be your issue:

conf t

interface Serial0/1/1:0
no fair-queue
end

wr mem

Billy

MasterComputing

ASKER

interface Serial0/1/1:0
no fair-queue
------
No luck

MasterComputing

ASKER

Billy, I sure appreciate your help. Here's the next output you requested:

HDORouter#show ppp multilink

Multilink1
Bundle name: 12.90.105.13
Remote Endpoint Discriminator: [2] 12.90.105.13
Local Endpoint Discriminator: [1] HDORouter
Bundle up for 00:08:38, total bandwidth 3072, load 1/255
Receive buffer limit 24000 bytes, frag timeout 1000 ms
0/0 fragments/bytes in reassembly list
0 lost fragments, 5 reordered
0/0 discarded fragments/bytes, 0 lost received
0x18E received sequence, 0x1EE sent sequence
Member links: 2 active, 0 inactive (max 255, min not set)
Se0/1/0:0, since 00:08:38
Se0/1/1:0, since 00:07:00
No inactive multilink interfaces
HDORouter#show ip cef
Prefix Next Hop Interface
0.0.0.0/0 attached Multilink1
0.0.0.0/8 drop
0.0.0.0/32 receive
12.90.105.12/30 attached Multilink1
12.90.105.12/32 receive Multilink1
12.90.105.13/32 attached Multilink1
12.90.105.14/32 receive Multilink1
12.90.105.15/32 receive Multilink1
12.175.223.224/29 attached GigabitEthernet0/0
12.175.223.224/32 receive GigabitEthernet0/0
12.175.223.225/32 receive GigabitEthernet0/0
12.175.223.227/32 attached GigabitEthernet0/0
12.175.223.231/32 receive GigabitEthernet0/0
127.0.0.0/8 drop
224.0.0.0/4 drop
224.0.0.0/24 receive
240.0.0.0/4 drop
255.255.255.255/32 receive
HDORouter#show ip int
GigabitEthernet0/0 is up, line protocol is up
Internet address is 12.175.223.225/29
Broadcast address is 255.255.255.255
Address determined by non-volatile memory
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
Input features: MCI Check
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
GigabitEthernet0/1 is administratively down, line protocol is down
Internet protocol processing disabled
Serial0/1/0:0 is up, line protocol is up
Internet protocol processing disabled
Serial0/1/1:0 is up, line protocol is up
Internet protocol processing disabled
Multilink1 is up, line protocol is up
Internet address is 12.90.105.14/30
Broadcast address is 255.255.255.255
Address determined by non-volatile memory
Peer address is 12.90.105.13
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is enabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF switching turbo vector
IP Null turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
Input features: MCI Check
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
HDORouter#

MasterComputing

ASKER

rfc1180, thanks for all your help. This was a screw up on the ISP's side. They had assigned me an in-use block of IPs for the GigabitEthernet side of the router.

MasterComputing

ASKER

Very prompt, useful information.

rfc1180

interesting; glad to hear that you got it fixed

MasterComputing

ASKER

Yes, interesting. Cost me 2 days of pulling out what little hair I have left. That's AT&T for you. Thanks again.