cisco ip sla issues

Hello Experts
i got two sites connected using WAN 100 mb point to point connection.
two cisco c3750x switches one at each site. (point to point line plugged into cisco)
two watch guard xtm firewall devices one at each site. (internet plugged into firewalls)
see attached diagram for details.
also attached switch configs for both switches.
during testing we take out internet cable for one site and can see switch routes traffic to internet via point to point line to other site internet line.
but this works only for around 30 sec then internet goes down for 30 sec.
again it starts working for 30 sec and then goes down so its continous loop.

not sure what I am doing wrong.
i want internet to failover also i use watchguard VPN tunnel to failover internal site routing if point to point link goes down.

hope someone can find the issue. thanks
Harry
Network-Sites.jpg
BHAM-cisco.txt
NTGM-Cisco.txt
LVL 3
H-SinghTechnical DirectorAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

InfamusCommented:
track 1 ip sla 1 reachability
 delay down 30 up 30
!
track 2 ip sla 2 reachability
 delay down 30 up 30


Remove "up 30"

So the statement will look like..

track 1 ip sla 1 reachability
 delay down 30 up 30
!
track 2 ip sla 2 reachability
 delay down 30
H-SinghTechnical DirectorAuthor Commented:
Hi infamous

I googled around and Cisco support forum says I need to put ip address in ip sla command
Mag be to tell it to only ping out via main site firewall

As upon failover it can again ping out via branch site and that makes track to go up down each 30 sec
InfamusCommented:
You already have ip setup as below...

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now
ip sla 2
 icmp-echo 172.16.100.2
 frequency 10
ip sla schedule 2 life forever start-time now
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

InfamusCommented:
Ahhhh...i see what is wrong.

You  only need one ip sla...

BHAM:

track 1 ip sla 1 reachability
 delay down 30

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now

ip route 10.0.0.0 255.255.255.0 172.16.100.2 track 1
ip route 10.0.0.0 255.255.255.0 172.16.10.10 10


NTGM

track 1 ip sla 1 reachability
 delay down 30

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now

ip route 0.0.0.0 0.0.0.0 172.16.50.10 track 1

ip route 0.0.0.0 0.0.0.0 172.16.100.1 10
H-SinghTechnical DirectorAuthor Commented:
actually i am monitoring two  line failovers, one is internet line failover second is point to point line failover so that I can send traffic out to firewall to use IPSEC tunnel , thats why we need two IP SLAs.
InfamusCommented:
Why don't you use dynamic routing for point to point?

That way your default route 0.0.0.0 can be routed to your internet.

When you lose connection to point to point it will go directly to the internet ipsec vpn.

Then you can use ip sla for the internet connection only which will be routed by the ip sla.
H-SinghTechnical DirectorAuthor Commented:
its getting bit complex for me as i also need watchguard to work with this whole setup, as i got NATed servers at both locations which need incoming ports mapping from both lines in case of failure to get incoming traffic mainly SIP trunks for phone servers. will be using BGP dynamic routing on Watchguard. but cisco needs to do all with IP SLA I believe.

or if you can advise better config please.
InfamusCommented:
You can configure OSPF on point to point routers only so it should advertise the routes between the two.

For example.

Main Office Router

router ospf 100
network 172.16.100.1/30 area 0
network 172.16.12.0/24 area 0
network 172.16.13.0/24 area 0
network 172.16.14.0/24 area 0
network 172.20.15.0/24 area 0

Branch Office Router
router ospf 100
network 172.16.100.2/30 area 0
network 10.0.0./24 area 0

BHAM:

track 1 ip sla 1 reachability
 delay down 30

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now

ip route 0.0.0.0 255.255.255.0 172.16.10.10 track 1
ip route 0.0.0.0 255.255.255.0 172.16.100.2 10


NTGM

track 1 ip sla 1 reachability
 delay down 30

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now

ip route 0.0.0.0 0.0.0.0 172.16.50.10 track 1

ip route 0.0.0.0 0.0.0.0 172.16.100.1 10
mikebernhardtCommented:
What are the IP addresses you are monitoring? Here (I believe) is the issue you're experiencing:
1. You have IP SLA to ping an IP address somewhere.
2. You disconnect cable, internet is lost, and IP address isn't reachable.
3. Internet connectivity switches over, internet is restored.
4. Now that IP address is also reachable, secondary internet drops again.
5. With no internet, IP address isn't reachable again.
6. Internet fails over again and is restored.
7. Repeat steps 4-6.

So, you need your IP SLA to point to an address that won't be reachable via the secondary internet. You can accomplish this by using an access list on the secondary interface that blocks traffic to that specific host address and allows everything else.
H-SinghTechnical DirectorAuthor Commented:
Hi Mike
on BHAM switch am using IPSLA Ping to 8.8.8.8
on NTGM switch am using IPSLA Ping to 4.4.2.2

Ideally we need to restrict this outgoing ping only to go out via onsite firewall
say BHAM ping must go only to 172.16.10.10 and not to re-route to any other
same for NTGM it needs to go via 172.16.50.10
then i believe it won't get response upon failure by using my point to point line.

am not familier with implementing ACLs but what if we do static route on both.
BHAM  ip route 8.8.8.8 255.255.255.0 172.16.10.10

NTGM ip route 4.4.2.2 255.255.255.0 172.16.50.10

will that restrict ping from switch to go only via given IP routes , then it shouldn't revert to second route.
H-SinghTechnical DirectorAuthor Commented:
Hi Mike further to your points.

1. You have IP SLA to ping an IP address somewhere.
Yes we pinging 8.8.8.8 from Main site and 4.4.2.2  from Brach office site.

2. You disconnect cable, internet is lost, and IP address isn't reachable.
yes say BHAM site cable disconnected so PC at bham have no internet now. ping broke

3. Internet connectivity switches over, internet is restored.
yes as with step3 ipsla switches over routes to P2P line. at this stage internet access is restored via Branch office internet line .

4. Now that IP address is also reachable, secondary internet drops again.
now ping starts again. as internet going out via second line so ipsla triggers event that track1 is up again as it can ping (even though not via Main internet line) so routes gets swiched and pointed back to main line which cable is still unplugged, so internet down for 30 sec,

5. With no internet, IP address isn't reachable again.
yes

6. Internet fails over again and is restored.
yes


7. Repeat steps 4-6.
yes
mikebernhardtCommented:
The static routes might work, you'd have to try it. But I would make it host-specific since your routes would force the entire /24 network to follow the static route. If that's OK, no problem. Otherwise,
BHAM  ip route 8.8.8.8 255.255.255.255 172.16.10.10
NTGM ip route 4.4.2.2 255.255.255.255 172.16.50.10

But static routes are kind of messy. An access list is easy. These block outbound icmp to the other site's SLA destination, but permit everything else:
NTGM
access-list 100 deny icmp any host 8.8.8.8
access-list 100 permit ip any any

interface [the outbound interface]
 ip access-group 100 out

BHAM
access-list 100 deny icmp any host 4.4.2.2
access-list 100 permit ip any any

interface [the outbound interface]
 ip access-group 100 out
H-SinghTechnical DirectorAuthor Commented:
Hello Guys
tried implementing ACLs.  
loop issue is fixed. so if I take cable out from BHAM internet line all my traffic diverts via P2P line out via NTGM internet line.  and this stays stable fine no disconnection each 30 sec. that issue is resolved.

but now am getting another issue. once traffic is diverted to second Site and i can see Track 1 shows reachability DOWN,  
if i plug cable back in for BHAM site internet.  that won't divert traffic back to BHAM line. as Trach 1 reachability stays DOWN.

any ideas please
H-SinghTechnical DirectorAuthor Commented:
I think i have managed to get this working now, by using static routes. so IP SLA ICMP Ping to external destinations is now restricted to go via same site internet line in that way its now can detect everything fine and brings Track status to up/down accordingly.

BHAM Site
ip sla 1
icmp-echo 4.2.2.1 source-ip 172.16.10.1
threshold 2
frequency 10
ip sla schedule 1 life forever start-time now
with a static route
ip route 4.2.2.1 255.255.255.255 172.16.10.10
----------------------------------------------------------

NTGM Site
ip sla 1
icmp-echo 4.2.2.2 source-ip 172.16.50.1
threshold 2
frequency 10
ip sla schedule 1 life forever start-time now
with a static route
ip route 4.2.2.2 255.255.255.255 172.16.50.10
-------------------------------------------------------------------------

Rest routes are done accordingly to use Tracks.

config is live in network and I shall monitor over holidays and confirm

thanks

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
mikebernhardtCommented:
Interesting on the access list solution, I hadn't thought about that issue! Once you cut over to the backup route, ping to the BHAM address would never work again even if the line came up, since all track was routed to the backup. The static route solution was the best way to go.
InfamusCommented:
H-Singh,

Is there a reason that you are avoiding the dynamic routing solution?

Adding static routes as needed may cause you a confusion.

OSPF is really good with inter-network routing which is also compatible with may other 3rd party devices.

If you are using all Cisco, you can also consider using EIGRP.

It is very convenient when it comes to auto failover.
H-SinghTechnical DirectorAuthor Commented:
Hi infamous
My skills with Cisco are basic
Its only these two switches I got Cisco
Firewalls next to these are watch guard which I beilive I need to do bgp dynamic routing to get incoming traffic working across both sites
This static route option right now worked for me
May be you can suggest an example for complete solution
As we also looking at failover of point to point line as well if p2p line fails I want cross site traffic to go via watchguard ipsec vpn tunnel
mikebernhardtCommented:
Using dynamic routing requires that both sides participate, and in this case the other side is the ISP. ISPs will not use OSPF or any IGP with a customer. The only other solution would be to use BGP as H-singh already mentioned. That's certainly possible but for this purpose it's overkill.
InfamusCommented:
@mike

Did you look at the diagram?

We are talking about the point to point connection and there's no ISP involved.
InfamusCommented:
@H-Singh

I already provided you with the complete step by step configuration above.

Please let me know if you have any questions.
H-SinghTechnical DirectorAuthor Commented:
thanks Infamus, i looked into ospf config, but i still need to use static route for ipsla 1 and ipsla1 config as i allready got now.

with ospf its only for local traffic and then i won't need ip sla 2 ??

i can certainly try that as well for P2P line with failover  via ipsec.
thanks
H-SinghTechnical DirectorAuthor Commented:
had to try various options and finally static route option worked for me.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Routers

From novice to tech pro — start learning today.