IP sla setup

So I have set up IP SLA on my router but I can't get it to rout out when the main line goes down. I can ping by IP address from the router but nothing else is getting out. My setup goes like this:
My network has 2 6509 switches running 4 vlans. I have one router with 2 ISP's coming in . 1 is direct connected with a 16 CIDR block of IPs.
The second is connected PPPoe with a 6 block of Ip's. I use route-maps to route the external IPs from both providers to the internal addresses and I use ACL's to route the traffic coming into all the IP's.  Attached is a network diagram and most of my running config. Again when I unplug the main line I can ping by address from the router but it will not resolve anything and no one can get out to the internet. I have no Idea what Im doing wrong.
EE_Config.txt
Network-Setup.jpg
LVL 1
Scott_Smith24Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Do you able to ping 8.8.8.8 , while you disconnected the main line?

If yes, have you tried to ping google.com from your router?
0
Scott_Smith24Author Commented:
Yes I can ping 8.8.8.8 but fails for www.google.com it say searching domain www.google.com then fails
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
can you remove the acl under dialer 1 and try. I don't see UDP port 53 allowed in that acl

I do see "permit tcp any host 33.33.33.14 eq domain", since you are using 8.8.8.8 as name server, either you need to add 8.8.8.8 in acl or remove acl and try
0
Top Threats of Q1 & How to Defend Against Them

WEBINAR: Join WatchGuard CTO and our Threat Research Team on Aug. 2nd to hear the findings from our Q1 Internet Security Report! Learn more about the top threats detected in the first quarter and how you can defend your business against them!

JustInCaseCommented:
When main line fails, is main default route
ip route 0.0.0.0 0.0.0.0 11.11.111.112 track 1
still in routing table?
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Predrag Jovic, while he disconnected the main line, he can ping 8.8.8.8

Hence, i assume the router has a default route via PPOE
0
Scott_Smith24Author Commented:
When the main line fails it read
Gateway of last resort is 33.33.33.15 to network 0.0.0.0
33.33.33.15 is next hop address on the backup ISP

Also in the Access List Inbound2 I do have permit tcp any host 33.33.33.14 eq domain
which is my wan interface IP on the secondary interface
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
While the router queries DNS, the source will be your router ip with random port and destination will be 8.8.8.8 with port 53,  

while return from 8.8.8.8, the source will be 8.8.8.8 with port 53. so u need to allow 8.8.8.8 under acl.

I do see another difference between your main interface acl and dialer interface acl is "permit udp any eq domain any"

permit udp any eq domain any  -- missed on the dialer interface acl
0
JustInCaseCommented:
Try to set

interface Dialer1
 ip mss 1462

on dial1 you are mising in your ACL Inbound2
 permit tcp any any established
 permit udp any eq domain any
as much as I can see.
0
Scott_Smith24Author Commented:
I did forget
permit tcp any any established
permit udp any eq domain any
at the end of and Access List Inbound2 I added it but wont be able to test it till tomorrow
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Cool.. we will wait for your further updates
0
Andrew DavisManagerCommented:
When disconnected main line, you can ping (from a client) 8.8.8.8 but not www.google.com that is a DNS error. Are your DNS servers internal and can you ping them?

Does the DNS server have access to an DNS forwarders? (perhaps you are using the main lines DNS, which may only resolve if coming from main line).

cheers
Andrew
0
Scott_Smith24Author Commented:
OK I added the 2 lines to the second ACL and the router can now ping by name and IP on the second ISP but my Lan is still not going out. I can't ping by name or IP. I can sucessfully ping the internal wan IP of the ISP but not the next hop. Here is my IP Brief when the main is taken off line

Gateway of last resort is 33.33.33.15 to network 0.0.0.0

S*    0.0.0.0/0 [10/0] via 33.33.33.15
      10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
S        10.110.109.159/32 [0/0], Virtual-Access2
C        10.110.110.0/24 is directly connected, GigabitEthernet0/1
L        10.110.110.1/32 is directly connected, GigabitEthernet0/1
S        10.110.111.0/24 [1/0] via 10.110.110.6
S        10.110.112.0/24 [1/0] via 10.110.110.6
S        10.110.113.0/24 [1/0] via 10.110.110.6
S        10.110.114.0/24 [1/0] via 10.110.110.6
      33.0.0.0/32 is subnetted, 1 subnets
C        33.33.33.15 is directly connected, Dialer1
      14.0.0.0/32 is subnetted, 1 subnets
C        33.33.33.14 is directly connected, Dialer1

So I can ping 33.33.33.14 but not 33.33.33.15
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
I believe the NAT table is not cleared


Can you run # Sh ip NAT translation
0
Scott_Smith24Author Commented:
That table is huge. Here are the basics. I would think I should at least be able to ping the next hop of the secondary IP 33.33.33.15

Pro Inside global         Inside local          Outside local         Outside global
--- 22.22.22.212        10.110.110.21         ---                   ---
--- 22.22.22.213        10.110.111.17         ---                   ---
--- 22.22.22.214        10.110.110.36         ---                   ---
--- 22.22.22.215        10.110.110.24         ---                   ---
--- 22.22.22.216        10.110.110.32         ---                   ---
--- 22.22.22.217        10.110.110.33         ---                   ---
--- 22.22.22.218        10.110.110.22         ---                   ---
--- 22.22.22.219        10.110.110.12         ---                   ---
--- 22.22.22.221        10.110.110.27         ---                   ---
--- 22.22.22.222        10.110.110.26         ---                   ---
--- 22.22.22.223        10.110.110.18         ---                   ---
--- 33.33.33.9          10.110.110.26         ---                   ---
--- 33.33.33.10         10.110.110.36         ---                   ---
--- 33.33.33.11         10.110.110.32         ---                   ---
--- 33.33.33.12         10.110.110.27         ---                   ---
--- 33.33.33.13         10.110.110.24         ---                   ---
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Let me explain in detail,

While the normal scenario all your LAN network will go thru primary link to reach internet after the NAT process

Once you unplugged the primary LAN cable, the NAT table still persists and NAT table shows next hop as your primary main link gateway.

To test your PPoE, you can add a specific NAT for anyone of LAN ip towards PPoE and the same ip should not get NAT via main link.

Add a route for 4.2.2.2 towards PPoE and try to ping from the testing new LAN ip to 4.2.2.2

If ping not working , check the NAT translation for the specific LAN ip

Sh ip NAT translation | i new LAN ip

Note:- you no need to disconnect the main line for the testing
0
Scott_Smith24Author Commented:
Sorry but how do I Add a route for 4.2.2.2 towards PPoE
0
Scott_Smith24Author Commented:
So I have a
ip nat inside source route-map nonatme interface GigabitEthernet0/0 overload
Do I need one for Dialer 1
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Yo test , you can a remote the NAT statement and add static NAT for the new LAN ip


ip nat inside source (new LAN ip) interface Dialer1
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Add a route for 4.2.2.2 towards dialed 1
0
Scott_Smith24Author Commented:
Im sorry I understand most everything but I don't know how to do what you are asking NetExpert Network Solutions Pte Ltd
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Sorry for that.. Let me explain with full configuration

allocate one test lan ip and assign the test lan ip to one of test laptop/pc

1. add a static route for 4.2.2.2 towards PPoE ==>  ip route 4.2.2.2 255.255.255.255 33.33.33.15
2. remove the current dialer1 nat statement add the below nat statement

no ip nat inside source route-map nonatme2 interface Dialer1 overload
ip nat inside source (test lan ip address) interface Dialer1 overload

3.Try ping 4.2.2.2 from the test laptop

4.If you are not able to ping 4.2.2.2 from the test laptop, please run the below command on the router and paste the output here
#sh ip route 4.2.2.2
#sh ip route <test lan ip address>
#sh ip nat translation | i test lan ip address
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Scott_Smith24Author Commented:
So that worked I ran
ip route 4.2.2.2 255.255.255.255 33.33.33.15
and
ip nat inside source static 10.110.110.15 interface Dialer1

I could ping 33.33.33.15 but nothing else
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
have you tried to ping 4.2.2.2 from the 10.110.110.15 system ?
0
Scott_Smith24Author Commented:
Yes that worked
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
Great. your PPoE is working fine as expected. Now we need to configure the internet link failvoer between primary(main link) to PPoE.

You can remove the recent config and put back all your old configuration.

As I said in my old thread that, In normal scenario all your traffic is going on with main line and if some issue on main link, the internet traffic should go over via PPoE. To achieve that, your old configurations are looks fine, but you need to write EEM script to clear the NAT table and routing table if the primary link fails



event manager applet ISP_DOWN
event track 1 state down
action 1.0 cli command “enable”
action 1.1 cli command “clear ip route *”
action 1.2 cli command “clear ip nat translation *”
action 1.3 cli command “end”
0
Scott_Smith24Author Commented:
OK I set that up but do I need a nat overload for Dialer 1
0
Scott_Smith24Author Commented:
So I have set that script but from 10.110.110.15 I still cant ping anything past 33.33.33.14. That pings fine but not the next hop which is directly connected to that 33.33.33.15. It seem like Im missing a route that tells my lan traffic to go out Dialer 1 if I run a traceroute it dies at my router interface of 10.110.110.1
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
have you added the nat overload command for Dialer1.  can you paste your current running config of router
0
Scott_Smith24Author Commented:
So I added the overload statement here is my abbv config


interface GigabitEthernet0/0
 description WAN Internet$ETH-WAN$
 ip address 11.11.111.112 255.255.255.240
 ip access-group Inbound in
 no ip redirects
 no ip unreachables
 ip nbar protocol-discovery
 ip flow ingress
 ip flow egress
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
!
interface GigabitEthernet0/1
 description Internal Network$ES_LAN$$ETH-LAN$
 ip address 10.110.110.1 255.255.255.0
 no ip redirects
 no ip unreachables
 ip flow ingress
 ip flow egress
 ip nat inside
 ip virtual-reassembly in
 duplex auto
 speed auto
 media-type rj45
 no mop enabled
!
interface GigabitEthernet0/2
 no ip address
 ip flow ingress
 ip flow egress
 duplex auto
 speed auto
 pppoe enable group global
 pppoe-client dial-pool-number 1
!
interface Virtual-Template1 type tunnel
 ip unnumbered GigabitEthernet0/0
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile CiscoCP_Profile1
!
interface Virtual-Template2
 ip unnumbered GigabitEthernet0/0
!        
interface Dialer1
 ip address negotiated
 ip access-group Inbound2 in
 ip mtu 1492
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 dialer pool 1
 ppp authentication chap callin
 ppp chap hostname ***********.net
 ppp chap password 7 *************
!
!
ip forward-protocol nd
!
ip http server
ip http access-class 1
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000
!
ip nat inside source route-map nonatme interface GigabitEthernet0/0 overload
ip nat inside source route-map nonatme2 interface Dialer1 overload
ip nat inside source static 10.110.110.92 22.22.22.211 route-map staticnatme
ip nat inside source static 10.110.110.21 22.22.22.212 route-map staticnatme
ip nat inside source static 10.110.111.17 22.22.22.213 route-map staticnatme
ip nat inside source static 10.110.110.36 22.22.22.214 route-map staticnatme
ip nat inside source static 10.110.110.24 22.22.22.215 route-map staticnatme
ip nat inside source static 10.110.110.32 22.22.22.216 route-map staticnatme
ip nat inside source static 10.110.110.33 22.22.22.217 route-map staticnatme
ip nat inside source static 10.110.110.22 22.22.22.218 route-map staticnatme
ip nat inside source static 10.110.110.12 22.22.22.219 route-map staticnatme
ip nat inside source static 10.110.110.27 22.22.22.221 route-map staticnatme
ip nat inside source static 10.110.110.26 22.22.22.222 route-map staticnatme
ip nat inside source static 10.110.110.18 22.22.22.223 route-map staticnatme
ip nat inside source static 10.110.110.26 33.33.33.9 route-map staticB
ip nat inside source static 10.110.110.36 33.33.33.10 route-map staticB
ip nat inside source static 10.110.110.32 33.33.33.11 route-map staticB
ip nat inside source static 10.110.110.27 33.33.33.12 route-map staticB
ip nat inside source static 10.110.110.24 33.33.33.13 route-map staticB
ip route 0.0.0.0 0.0.0.0 11.11.111.112 track 1
ip route 0.0.0.0 0.0.0.0 33.33.33.15 10
ip route 10.110.109.0 255.255.255.0 GigabitEthernet0/0
ip route 10.110.111.0 255.255.255.0 10.110.110.6
ip route 10.110.112.0 255.255.255.0 10.110.110.6
ip route 10.110.113.0 255.255.255.0 10.110.110.6
ip route 10.110.114.0 255.255.255.0 10.110.110.6
!
ip access-list extended Inbound
 permit tcp any host 11.11.111.111 eq 443
 permit tcp any host 11.11.111.111 eq 22
 permit gre any host 11.11.111.111
 permit icmp any host 11.11.111.111 unreachable
 permit icmp any host 11.11.111.111 echo
 permit icmp any host 11.11.111.111 echo-reply
 permit icmp any host 11.11.111.111 packet-too-big
 permit icmp any host 11.11.111.111 time-exceeded
 permit icmp any host 11.11.111.111 traceroute
 permit icmp any host 11.11.111.111 administratively-prohibited
 permit udp any host 11.11.111.111 eq non500-isakmp
 permit udp any host 11.11.111.111 eq isakmp
 permit esp any host 11.11.111.111
 permit ahp any host 11.11.111.111
 permit tcp any host 11.11.111.111 eq domain
 permit udp any host 11.11.111.111 range 16399 16472
 permit esp any any
 permit tcp any any eq 10000
 permit udp any eq ntp any
 permit udp any any eq isakmp
 permit udp any any eq non500-isakmp
 permit tcp any host 22.22.22.211 eq 4073
 permit tcp any host 22.22.22.212 range 50000 50010
 permit tcp any host 22.22.22.212 eq ftp-data
 permit tcp any host 22.22.22.212 eq ftp
 permit tcp any host 22.22.22.213 eq 8080
 permit tcp any host 22.22.22.213 eq 443
 permit tcp any host 22.22.22.213 eq 5222
 permit tcp any host 22.22.22.223 eq www
 permit tcp any host 22.22.22.212 eq 16080
 permit tcp any host 22.22.22.214 eq smtp
 permit tcp any host 22.22.22.215 eq 81
 permit tcp any host 22.22.22.215 eq www
 permit tcp any host 22.22.22.215 eq 443
 permit tcp any host 22.22.22.215 eq 465
 permit tcp any host 22.22.22.215 eq 143
 permit tcp any host 22.22.22.215 eq 5233
 permit tcp any host 22.22.22.215 eq 636
 permit tcp any host 22.22.22.215 eq 389
 permit tcp any host 22.22.22.215 eq 993
 permit tcp any host 22.22.22.216 eq www
 permit tcp any host 22.22.22.221 eq www
 permit tcp any host 22.22.22.221 eq 16880
 permit tcp any host 22.22.22.216 eq 16080
 permit tcp any host 22.22.22.217 eq www
 permit tcp any host 22.22.22.217 eq 16080
 permit tcp any host 22.22.22.218 eq 22350
 permit tcp any host 22.22.22.218 eq 6005
 permit tcp any host 22.22.22.218 eq 18008
 permit tcp any host 22.22.22.218 eq www
 permit tcp any host 22.22.22.218 eq 8080
 permit tcp any host 22.22.22.218 eq 19001
 permit tcp any host 22.22.22.222 eq 6005
 permit udp any host 22.22.22.222 eq 8080
 permit tcp any host 22.22.22.222 eq 19001
 permit tcp any host 22.22.22.222 eq 18008
 permit tcp any host 22.22.22.222 eq www
 permit tcp any host 22.22.22.222 eq 16001
 permit udp any host 22.22.22.219 eq non500-isakmp
 permit udp any host 22.22.22.219 eq 1701
 permit udp any host 22.22.22.219 eq isakmp
 permit tcp any host 22.22.22.219 eq www
 permit tcp any host 22.22.22.219 eq 4500
 permit tcp any any established
 permit udp any eq domain any
ip access-list extended Inbound2
 permit tcp any host 33.33.33.14 eq 443
 permit tcp any host 33.33.33.14 eq 22
 permit gre any host 33.33.33.14
 permit icmp any host 33.33.33.14 unreachable
 permit icmp any host 33.33.33.14 echo
 permit icmp any host 33.33.33.14 echo-reply
 permit icmp any host 33.33.33.14 packet-too-big
 permit icmp any host 33.33.33.14 time-exceeded
 permit icmp any host 33.33.33.14 traceroute
 permit icmp any host 33.33.33.14 administratively-prohibited
 permit udp any host 33.33.33.14 eq non500-isakmp
 permit udp any host 33.33.33.14 eq isakmp
 permit esp any host 33.33.33.14
 permit ahp any host 33.33.33.14
 permit tcp any host 33.33.33.14 eq domain
 permit udp any host 33.33.33.14 range 16399 16472
 permit esp any any
 permit tcp any any eq 10000
 permit udp any eq ntp any
 permit udp any any eq isakmp
 permit udp any any eq non500-isakmp
 permit tcp any host 33.33.33.9 eq 6005
 permit udp any host 33.33.33.9 eq 8080
 permit tcp any host 33.33.33.9 eq 19001
 permit tcp any host 33.33.33.9 eq 18008
 permit tcp any host 33.33.33.9 eq www
 permit tcp any host 33.33.33.9 eq 16001
 permit tcp any host 33.33.33.10 eq smtp
 permit tcp any host 33.33.33.12 eq www
 permit tcp any host 33.33.33.12 eq 16880
 permit tcp any host 33.33.33.13 eq 81
 permit tcp any host 33.33.33.13 eq www
 permit tcp any host 33.33.33.13 eq 443
 permit tcp any host 33.33.33.13 eq 465
 permit tcp any host 33.33.33.13 eq 143
 permit tcp any host 33.33.33.13 eq 5233
 permit tcp any host 33.33.33.13 eq 636
 permit tcp any host 33.33.33.13 eq 389
 permit tcp any host 33.33.33.13 eq 993
 permit tcp any host 33.33.33.11 eq www
 permit tcp any host 33.33.33.11 eq 16880
!
ip sla 1
 icmp-echo 11.11.111.112 source-interface GigabitEthernet0/0
 threshold 2
 timeout 1000
 frequency 3
ip sla schedule 1 life forever start-time now
logging trap debugging
!
route-map staticnatme permit 10
 match ip address 105
!
route-map nonatme permit 10
 match ip address 101
!
route-map nonatme2 permit 10
 match ip address 101
!
!
snmp-server community TestCommunity RO
snmp-server ifindex persist
access-list 1 remark HTTP Access-class list
access-list 1 remark SDM_ACL Category=1
access-list 1 permit 10.110.110.0 0.0.0.255
access-list 1 deny   any
access-list 23 permit 10.10.10.0 0.0.0.7
access-list 100 remark VTY Access-class list
access-list 100 remark SDM_ACL Category=1
access-list 100 permit ip 10.110.110.0 0.0.0.255 any
access-list 100 deny   ip any any
access-list 101 deny   ip 10.110.110.0 0.0.0.255 10.110.111.0 0.0.0.255
access-list 101 deny   ip host 10.110.110.21 any
access-list 101 deny   ip host 10.110.110.18 any
access-list 101 deny   ip host 10.110.110.22 any
access-list 101 deny   ip host 10.110.110.32 any
access-list 101 deny   ip host 10.110.110.36 any
access-list 101 deny   ip host 10.110.110.33 any
access-list 101 deny   ip host 10.110.111.17 any
access-list 101 deny   ip host 10.110.110.24 any
access-list 101 deny   ip host 10.110.110.27 any
access-list 101 deny   ip host 10.110.110.26 any
access-list 101 permit ip 10.110.110.0 0.0.0.255 any
access-list 101 permit ip 10.110.112.0 0.0.0.255 any
access-list 101 permit ip 10.110.111.0 0.0.0.255 any
access-list 101 permit ip 10.110.113.0 0.0.0.255 any
access-list 101 permit ip 10.110.114.0 0.0.0.255 any
access-list 105 deny   ip 10.110.110.0 0.0.0.255 10.110.112.0 0.0.0.255
access-list 105 permit ip 10.110.110.0 0.0.0.255 any
access-list 105 permit ip 10.110.111.0 0.0.0.255 any
access-list 105 permit ip 10.110.112.0 0.0.0.255 any
access-list 106 deny   ip 10.110.110.0 0.0.0.255 10.110.112.0 0.0.0.255
access-list 106 permit ip 10.110.110.0 0.0.0.255 any
access-list 106 permit ip 10.110.111.0 0.0.0.255 any
access-list 106 permit ip 10.110.112.0 0.0.0.255 any
access-list 150 remark CCP_ACL Category=16
access-list 150 permit ip 10.110.111.0 0.0.0.255 10.110.109.0 0.0.0.255
access-list 150 permit ip 10.110.110.0 0.0.0.255 10.110.109.0 0.0.0.255
access-list 150 permit ip 10.110.112.0 0.0.0.255 10.110.109.0 0.0.0.255
!
!
!
control-plane
!
 !
 !
 !
 !
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
!
!        
!
!
!
gatekeeper
 shutdown
!
!
event manager applet ISP_DOWN
 event track 1 state down
 action 1.0 cli command "enable"
 action 1.1 cli command "clear ip route*"
 action 1.2 cli command "clear ip nat translation *"
 action 1.3 cli command "end"
!
!
end
0
Scott_Smith24Author Commented:
Ok so no matter what I do I cant get my clients to get out to the internet. Servers are fine but clients cant even ping the next hop off the dialer 2. What am I doing wrong?
0
NetExpert Network Solutions Pte LtdTechnical SpecialistCommented:
suspecting something on the dialer acl.

can you remove the dialer acl and try.. this is just to isolate the acl from the troubleshooting methods
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Routers

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.