Cisco 2811 config with Verizon Bonded T1 lines

controller T1 0/0/0
  framing esf
  clock source internal
  linecode b8zs  <---- that is common, but you need to verify if Verizon is using that
  channel-group 0 timeslots 1-24  <---  I think this will be more for you since you have bonded
bridge irb
!
!
interface FastEthernet0/0
  no ip address
  no ip redirects
  no ip proxy-arp
  ip nbar protocol-discovery
  ip flow ingress
  ip flow egress
  ip route-cache flow
  ip tcp adjust-mss 1412
  duplex full
  speed 100
  no mop enabled
  bridge-group 1
!
!
interface Serial0/0/0:0
  no ip address
  bridge-group 1
!
!
bridge 1 protocol ieee
bridge 1 route ip

That is basically what I have on my 2811 routers.  I have them bridged, though, so the settings may differ for your need.

What is the difference between bridging and bonding? I need this pipe to act as a 3Mb line.

I think what the preceding answer is referring to is that he has the FastEthernet and Serial ports bridged so there is just a Layer 2 connection between them and everything that appears on one appears on the other.  This is a very common setup with wireless in Cisco products or in other scenarios where you want to push traffic across dissimilar interfaces (which is what is being done in the example above).

Channel bonding, on the other hand, is combining two similar interfaces into one larger pipe and it sounds like that is what you want to do.   What you probably need is a configuration for PPP Multi-Link and that configuration would look something like this:

Serial0/0
framing esf
linecode b8zs
channel-group 0 timeslots 1-24
!
Serial0/1
framing esf
linecode b8zs
channel-group 0 timeslots 1-24
!
interface Multilink1
ip address 192.168.254.1 255.255.255.252
no cdp enable
ppp multilink
ppp multilink group 1
!
interface Serial0/0
no ip address
encapsulation ppp
no fair-queue
ppp multilink
ppp multilink group 1
!
interface Serial0/1
no ip address
encapsulation ppp
no fair-queue
ppp multilink
ppp multilink group 1
!

Each additional interface can be added with another iteration of the "interface Serial0/x" code block and the "Serial0/x" block.

However, it is possible that your provider is using a multi-link ATM circuit or something else weird.  But if the circuits support PPP MultiLink, this type of configuration may be close to what you need.

I think they are using Frame Relay technology. This is the example Verizon sent me, but they could not send me one for bonding the circuits. Do you know what the config would be when bonding frame circuits?

!
version 12.1                                    ! IOS Version
service timestamps debug uptime                 ! Requests timestamp of
logs
no service password-encryption                  ! No password encryption
!                                               ! Off by default
hostname ROUTER
!
ip subnet-zero                                  ! Allows the router to
use the first subnet of a block
ip cef                                          ! Enables Cisco express
forwarding
no ip finger                                    ! Disables finger on the
router (UNIX Comand)
!
interface FastEthernet0/0                       ! Customer Interior
Interface
ip address XX.XX.XX.XX 255.255.255.0            ! Interior IP Address
no  ip mroute-cache                             ! Disables multicast
fast
switching
no keepalive                                    ! Disables Ethernet
keepalives
speed 100                                       ! Locks down port speed
full-duplex                                     ! Locks down duplex mode
no cdp enable                                   ! Disable Cisco
Discovery Protocol
!
interface Serial0/0                             ! Customer WAN interface
no ip address
encapsulation frame-relay IETF                  ! Sets interface
encapsulation
service-module t1 timeslots 1-24                ! Sets timeslots
(channels of T1)
service-module t1 framing esf                   ! Sets T1 Framing
service-module t1 linecode b8zs                 ! Sets T1 Linecode
service-module t1 clock source internal         ! Sets clocking to
Internal
frame-relay lmi-type ansi                       ! Set LMI type
load-interval 30                                ! Sets length of time
load
calculations
no fair-queue                                   ! Enable Fair Queuing on
an Interface
bandwidth                                       ! Bandwidth. Just a
description
!
interface Serial0/0.1 point-to-point            ! Enables sub-interface
ip addresss XX.XX.XX.XX 255.255.255.252         ! IP address of WAN
interface
frame-relay interface-dlci 500 IETF             ! Sets DLCI and
Encapsulation
no ip redirects                                 ! Disables redirect
messages from WAN INT.
no ip proxy-arp                                 ! Disables ARP per
RFC1027
no arp frame-relay                              ! Disables ARP for the
interface
no cdp enable                                   ! Disable Cisco
Discovery Protocol
bandwidth                                       ! Bandwidth. Just a
description
!
ip route 0.0.0.0 0.0.0.0 Serial0/0.1            ! Sets default gateway
of last esort
ip classless                                    ! Enable classless
routing behavior
no cdp run                                      ! Disable Cisco
Discovery Protocol
no ip http server                               ! Disables HTTP server
feature (for security)
!
ip domain-name ALTER.NET                        ! Sets domain-name
ip name-server 198.6.1.1                        ! DNS server router uses
ip name-server 198.6.1.2                        ! Secondary DNS server
router uses
snmp-server community 6f270ca640 RO             ! SNMP community string
snmp-server trap-authentication
!

I guess the question then becomes whether a Frame Relay sub-interface can participate in a multi-link group and I don't have an immediate answer to that.  I  will try to come up with an answer for you but if someone has a working configuration for that, I will gladly yield to them for the answer.

After surfing over to the Nortel site to see what they might be using in the existing solution, I started wondering if Verizon might be using Nortel's Split Multi-Link Trunking (or Multi-Link Split Trunking).  If so, you might be looking at a proprietary solution that you will not be able to duplicate with your Cisco router.  Any chance you could get the model number from your Nortel CPE and post it?

I was able to get the config from my Nortel router that does work with this circuit. I am just unsure how to translate it to Cisco speak:

module  t1 1
    clock_source line
    exit t1
module  t1 2
    clock_source line
    exit t1
interface  ethernet 0
    ip  address <LAN IP> 255.255.255.192
    qos
      exit qos
    exit ethernet

interface  bundle MFR1
    link  t1 1
    link  t1 2
    encapsulation frelay
    fr
      intf_type dte
      frame_size 4470
      mfr  fragment_size 4000
      mfr  seg_threshold 3999
      lmi ansi
        exit lmi
      pvc 500
        no frf12
        shaping cir 3072000 bcmax 3072000 bcmin 65536
        no policing
        ip  address <WAN IP> 255.255.255.252
        map <WAN IP GATEWAY AT VERIZON>
        red
          exit red
        qos
          exit qos
        exit pvc
      interleave
        hiprio 50 100
        exit interleave
      exit fr
    qos
      exit qos
    exit bundle
hostname *******
log utc
telnet_server
system  display-boot-config no
system  hdlc_error 6000
ip
  load_balance per_flow
  route 0.0.0.0 0.0.0.0 <WAN IP GATEWAY AT VERIZON> 1
  exit ip

I'm thinking that the implementation your router is showing is Nortel proprietary and depends on Nortel equipment on the other end, but I will confirm that after I get home and get some dinner.

Thank you

Well, it appears I was wrong - what you need is "multi-link frame relay" and the config should look something like this (from Cisco's Web site):

interface MFR1
   no ip address
   mls qos trust dscp
   frame-relay intf-type dce
   frame-relay multilink bid router1
!
interface MFR1.1 point-to-point
   ip address 10.0.1.1 255.255.255.0
   ip pim sparse-mode
   mls qos trust dscp
   frame-relay interface-dlci 100

interface Serial5/0
   encapsulation frame-relay MFR1
   frame-relay multilink lid first-link
   frame-relay multilink hello 9
   frame-relay multilink retry 3

interface Serial6/0
   encapsulation frame-relay MFR1
   frame-relay multilink ack 4

The Cisco IOS Wide-Area Networking Configuration Guide Release 12.4T has all the details and now that you've got me interested, I'm going to take a look there and see if any caveats pop up.  You can find it at:

http://www.cisco.com/en/US/products/ps6441/products_installation_and_configuration_guides_list.html

Beware, they have rearranged their Web site and are doing away with the old UniverCD and CCO Website in favor of a new and improved document library.  But many of the references are now available in a book-length PDF file which is very handy for keeping on the laptop to take from site to site.

After reading through the WAN Command Reference, it looks as though the skeleton configuration above should be close to workable.  Unfortunately, I don't have my lab set up to test frame-relay any more since my WAN is now all Ethernet-format microwave.  But I hope this gives you a start towards the solution.

pmwrightjr,
good work! Welcome to EE. Hope to see you around in other threads. There's nothing like the "good answer" emails, is there?

pmwrightjr:  Thanks so much for picking this up.  I wasn't able to respond back due to work issues on my own network. :-(  I'm glad you were able to take off on where I started and fill the author in on where I was going. :)

You're quite welcome but it reading back over my responses it was more like a stream-of-consciousness journey of discovery sort of thing rather than a coherant answer.  But I eventually got there, I guess  :-)

You know....that is what is great about collaboration.  It allows discussion of thoughts which ultimately provides the solution.  I'm curious if HCCI got this working.

After reading this posting and finding another EE string that kind of helped:

http://www.experts-exchange.com/Hardware/Networking_Hardware/Routers/Q_22468805.html

....I am going to deploy the below config tonight. If you happen to notice anything wrong with it, please let me know.

interface MFR1
 description :MLFR:NxT1
 mtu 4470
 bandwidth 3072
 no ip address
 no ip redirects
 no ip proxy-arp
 no ip mroute-cache
 load-interval 30
 no arp frame-relay
 frame-relay lmi-type ansi
!

interface MFR1.500 point-to-point
 ip address <Public WAN IP> 255.255.255.252
 ip access-group 100 in
 no ip redirects
 no ip proxy-arp
 no arp frame-relay
 no cdp enable
 frame-relay interface-dlci 500 IETF


!
interface FastEthernet0/0
 description $ETH-LAN$$ETH-SW-LAUNCH$$INTF-INFO-FE 0/0$
 ip address <Public LAN IP> 255.255.255.192
 duplex full
 speed auto

!
interface Serial0/0/0
 description :MLFR:NxT1
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 no arp frame-relay
!
interface Serial0/1/0
 description :MLFR:NxT1
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 no arp frame-relay
!
no ip classless
ip route 0.0.0.0 0.0.0.0 <WAN GW>

Just remember that the gateway must be to the next physically connected hop.  So if your <WAN GW> is directly connected to this 2811, you are set for the ip route 0.0.0.0 0.0.0.0 routing.  The one thing I would change, and this is per the ICND class I took, is that you will want to set the speed on your FastEthernet0/0 to the supported top speed on both ends.  Mine are connected to a 10/100 switch so I set mine to Full Duplex and 100 on both ends.  While devices are supposted to auto negotiate, I would not trust that for switches and routers.  For your desktops, go ahead and use auto negotiate, but since this is a router connecting (I presume) to a switch, I would set the port speed and duplex.

OK, deployed the confirm and it seems we are half way there. Verizon can PING my WAN IP but not my LAN Public IP. I can PING 4.2.2.2 from the router, but not from the FW that is plugged into it.

It's like it is not able to route from the LAN interface to the WAN and via versa.

What happens if you do an extended ping using the LAN address as the source IP and vice-versa?  Can you ping the other interface within the router?
Can you ping a loopback interface?

Can you post a routing table from the router?

And although it might be too obvious, have you enabled IP routing with an 'ip routing' statement? (Don't laugh - I actually forgot that once on a border router and was baffled for the longest time)

What is 4.2.2.2?

Am I asking too many questions?  :-)

HCCI, you are more than halfway.  This part is actually a lot easier to get going as it is simply a routing issue on your router to inform traffic how to get to your LAN.

So to get this right.  When you telnet into your 2811 router, you can ping a known pingable address such as yahoo.com (206.190.60.37)?  When you try pinging that from within the LAN, you are not able to ping it?  If you can ping by the address, then it is a DNS issue (but I doubt this is your issue). BTW, what is FW?  Can you post your full config file?  It would be helpful if you could post your non-routable IP addresses as well.  If you do not feel comfortable with that, then use a similar model with different IPs.  It makes it hard to figure out a routing issue if the IPs are not listed since you may not have your routing configured properly for your subnet.  Such as using a default gateway that is 2 hops away.  This will result in failed pings.

In reading through this whole thread, I did find one little clue that may or may not mean anything.  
In the sample configuration that you got from Verizon, they enable classless routing with an 'ip classless' statement.  In your configuration, you disable classless routing with the second-to-last statement.
In view of this, I think it would be particularly important to take a very close look at the routing table because you could have something like a 4.2.2.2/30 address being treated like a 4.2.2.2/8 classful address and that could definitely push your routing decisions over the edge.

I'm not sure, pmwrightjr.  If that was the case, I would think the router wouldn't get to it as well.  It seems to be an issue with the LAN hitting the router and then not knowing what to do with the traffic from the LAN.  If you run a tracert from one of your workstations to yahoo.com, where does it hang?

You can try what pmwrightjr said and put it into ip classless and see if it works.

Here is my entire config. I have remove password and encryption information and changed the IP, but the IP scheme represents one very similar to mine.

Current configuration : 3791 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname HCC_ROUTER
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
!
no aaa new-model
!
resource policy
!
ip subnet-zero
!
!
ip cef
!
!
ip domain name yourdomain.com
ip name-server 198.6.100.53
ip name-server 198.6.100.25
ip name-server 4.2.2.2
!
!
!
crypto pki trustpoint TP-self-signed-*******
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-*******
 revocation-check none
 rsakeypair TP-self-signed-*******
!
!
crypto pki certificate chain TP-self-signed-*******
 certificate self-signed 01
  <HEX CODES>
  quit
username admin privilege 15 secret 5 <Password>
!
!
!
!
!
interface MFR1
 description :MLFR:NxT1
 mtu 4470
 no ip address
 no ip redirects
 no ip proxy-arp
 no ip route-cache cef
 no ip mroute-cache
 load-interval 30
 frame-relay lmi-type ansi
!
interface MFR1.500 point-to-point
 ip address 137.235.18.142 255.255.255.252
 ip access-group 100 in
 no ip redirects
 no ip proxy-arp
 no cdp enable
 frame-relay interface-dlci 500 IETF
!
interface FastEthernet0/0
 description $ETH-LAN$$ETH-SW-LAUNCH$$INTF-INFO-FE 0/0$
 ip address 69.21.189.129 255.255.255.192
 duplex full
 speed auto
!
interface FastEthernet0/1
 ip address 192.168.21.1 255.255.255.0
 shutdown
 duplex auto
 speed auto
 no routing dynamic
!
interface Serial0/0/0
 description :MLFR:NxT1
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 service-module t1 timeslots 1-24
 no arp frame-relay
!
interface Serial0/1/0
 description :MLFR:NxT1
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 service-module t1 timeslots 1-24
 no arp frame-relay
!
ip classless
ip route 0.0.0.0 0.0.0.0 137.235.18.141
!
ip http server
no ip http secure-server
!
!
!
control-plane
!
!
!
line con 0
 login local
line aux 0
line vty 0 4
 access-class 23 in
 privilege level 15
 password 7 <password>
 login
 transport input telnet ssh
line vty 5 14
 access-class 23 in
 privilege level 15
 login local
 transport input telnet ssh
line vty 15
 access-class 23 in
 privilege level 15
 password 7 <password>
 login local
 transport input telnet ssh
!
scheduler allocate 20000 1000
!

Also, Verizon sent me this config last night, which I tried today but my router did not like the "controller T1 0/0/0" command.

Appendix A.  CPE Configurations
!
controller T1 0/0/0
 framing esf
 linecode b8zs
 channel-group 0 timeslots 1-24 speed 64 !
controller T1 0/0/1
 framing esf
 linecode b8zs
 channel-group 0 timeslots 1-24 speed 64 !
interface MFR1
 mtu 4470
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay IETF
 no ip route-cache cef
 no ip mroute-cache
 load-interval 30
 no arp frame-relay
 frame-relay lmi-type ansi
 frame-relay multilink bid to gw
!
interface MFR1.500 point-to-point
 ip address x.x.x.2 255.255.255.252
 no ip redirects
 no ip proxy-arp
 no arp frame-relay
 no cdp enable
 frame-relay interface-dlci 500 IETF  
!        
interface FastEthernet0/0
 ip address a.a.a.1 255.255.255.0
!
interface Serial0/0/0:0
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 no arp frame-relay
!
interface Serial0/0/1:0
 mtu 4470
 bandwidth 1536
 no ip address
 no ip redirects
 no ip proxy-arp
 encapsulation frame-relay MFR1
 no arp frame-relay
!
!
ip route 0.0.0.0 0.0.0.0 MFR1.500

I notice an ACL on the input (ip access-group 100 in) - is this access-list and the access-list for the TTY ports (access-list 23) defined in a part of the configuration you didn't send.  It could be relevant if pings are being denied as is often the case.

The "controller" terminology is for a different type of interface hardware which is why your router didn't like it.  My 2811 with a dual-port MFT controller (VWIC-2MFT-T1) uses the terminology that Verizon proposed in their latest note but my 2621XM routers with a WIC-1DSU-T1 use only the "serial" terminology.  So you have that part right with the config from last night, you just have a routing issue and it's not likely that the interface configuration has anything to do with it.

I wish I had a brilliant flash of inspiration for you but looking at the ACLs is the only thing I can offer at this point.  The output from "sh ip route" would be very helpful at this point, even if you wanted to redact all the routable (i.e. non-RFC1918) addresses.

What I sent you above is my entire config. So should I remove the "ip access-group 100 in" command from the MFR interface?

Anything else that sticks out? The routing in not complicated (only one route) so I don't really understand how it came be a routing issue.

Hmm... I dont know what the effect of having an "access-group" statement without having the associated access-list, but yes, you might try removing it and see what happens.  Eventually, you will probably want it as a security measure - typically an access list in that position is used to block any traffic except that which is headed for your LAN.  When you get to that point, Cisco has an excellent document on constructing an ACL for Internet egress but for now, I think we need to eliminate that as a possibility.
Once you try that, if you could review again what you can ping and what you can't.  At this point, Verizon can ping your WAN interface but not your LAN?  From a workstation on the LAN, you can ping the LAN but not the WAN interface and not an address beyond the WAN?  Is that where we are?  If not, let me know!

It might not be a routing issue but a routing table can sometimes give valuable clues about where the packets might be going.  Also, pings fail with different codes and those can be helpful (if it fails with a 'U', it means the host is not reachable by that route, if it fails with a period, it indicates that the network server timed out waiting for a replay).

I think the reason we are thinking about routing issues that is the fact that Verizon can ping your WAN interface, so that indicates the circuit is up.  But if you wanted to be sure of that, you can do a "show frame-relay multilink" and check the output.
The extended ping (type 'ping' and hit enter with no address to step through the choices one by one) can also be particularly valuable because you can determine the source of the packet).
If you want to see more detail, you can do a ping debug with 'debug ip icmp' to see the ping details as they are sent and received.  If you do this through a telnet session, be sure to do a 'term mon' so you see the output and make sure you turn off all debugging when you are through  with  an 'undebug all'.

If you still have some thought that it might be a frame relay problem, you can try turning up a frame-relay debug and watching the lmi packets come and go or watch the circuit setup, etc.  A good one to start would be 'debug frame-relay multilink control mfr1'

I think you're close... very, very close but you need to somehow obtain some visibility into what the packets are actually doing as they enter and traverse the router.  That's why I recommend the routing table, followed by various ping tests including extended pings from different sources and then finally some debug sessions.

For example, if you were doing a ping debug, you would most likely see exactly what is happening with the pings from your LAN (e.g. whether they are entering the router but don't have a route back or whether they are not entering at all).

The other suggestion I would make is to create a loopback interface with an address used nowhere else on your LAN or WAN - maybe something like this:

interface Loopback 0
  ip address 172.16.254.254 255.255.255.255
  no shutdown

That gives you an anchor point that you can use to test connectivity with one interface at a time and a loopback interface is always up.  There are other uses for loopback interfaces but they are outside the scope of what we are doing here but there is no harm in creating one.