Solved

Network Failover Steps

Posted on 2011-02-18
8
338 Views
Last Modified: 2012-06-21
Hello,

From a network administrator's perspective, what steps should be taken to either test a failover or go through one actually?

For the sake of simplicity, we'll consider an HO and a DR location equipped with the same equipment. The DR is hot with data being replicated. I've attached an ascii representation of the network.

What step by step procedures must be in place to expedite a smooth failover?

You don't have to write it up yourselves, just point me to the links or documents that'll help me get the answers.

Thanks for looking.
ASCII-Network.JPG
0
Comment
Question by:netcmh
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
8 Comments
 
LVL 20

Author Comment

by:netcmh
ID: 34926439
Cases, not complete by any means, but a start:

1. If the internet connection at the active fails, the internet connection at the standby should take over and service requests, both internal and external.

2. If the internet router at the active fails, the internet router at the standby should take over and service requests, both internal and external.

3. If the ASA at the active site fails, the ASA at the standby site should take over and service requests.

4. If the ISA at the active site fails, the ISA at the standby should take over and service requests.

5. If the connection between the active and standby sites fails, branches should be able to communicate via the MPLS to the active site.

6. If the MPLS router at the active site fails, the MPLS router at the standby site should be able to route traffic from the branches to the active site.

I can't figure out exactly what steps to take or the procedure to test the failover.

Any and all input would be highly appreciated.
0
 
LVL 20

Author Comment

by:netcmh
ID: 34926570
I'm willing to open another ticket for those on the fence - points-wise. I need help on this - bad!

Thanks
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 34927217
>From a network administrator's perspective, what steps should be taken to either test a failover or go through one actually?

Typically through a maintenace window, there is no such thing as a smooth failover (Typically) at layer 2 or 3 depending on the protocols that you are using. The process is stright forward, just start pulling cables and monitoring the failover.

>Cases, not complete by any means, but a start:
All your cases are strong and are assumptions based on the configuration of the equipement and protocols, so with that being said, start testing at the ISP level by disconnecting one cable and testing failover. But, honestly, a realistic recommendation can not be given based on what you have without additional information (Strong Network Diagram, configs, etc).

Billy
0
Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

 
LVL 20

Author Comment

by:netcmh
ID: 34928366
I was hoping for a general procedure, not necessarily a custom made one to suit me.

Pulling cables or turning off the ports would be the ideal way to test the failover theory. Of course, dynamic routing at the core helps. Vlans trunked across the link to the DR helps keep the segments segregated.

I know all this. But, how does one go about documenting it? What procedures need to be implemented?

Thanks Billy.
0
 
LVL 20

Author Comment

by:netcmh
ID: 34942608
Anyone?
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 34945849
>I was hoping for a general procedure, not necessarily a custom made one to suit me.
That is the idea, there is not a general procudure that fits all; depending on the protocols that you have used in your network will dictate on how the network will fail-over and reconverge. Once you understand your network, you can build out the procedure. If you understand the protocols and traffic flows, etc, you will understand how the failover will occur; this is also usually a step in the design process.

Billy
0
 
LVL 20

Author Comment

by:netcmh
ID: 34951461
Billy,

Can you give me something more concrete?

Steps to test routing, verify connectivity, change configs - anything?
0
 
LVL 24

Accepted Solution

by:
rfc1180 earned 500 total points
ID: 34953360
>Steps to test routing
use traceroute to verify hops in the path of your network, are you taking the predicted path?

>verify connectivity
Ping servers at a specific location

>change configs
No need to change the configs during a failover; if anything, you log in via the CLI and maybe shutdown an interface to simulate a cable break.

Save configs for all devices in the network as some will require to power down and power up.

1. If the internet connection at the active fails, the internet connection at the standby should take over and service requests, both internal and external.

Test by pulling the cable from the ISP side, use traceroute and ping to verify connectivity to google.com. Troubleshoot as necessary if testing fails. This might require tracking objects and/or utilizing IP SLA. Check the status of the standby group, check the track objects, etc.


2. If the internet router at the active fails, the internet router at the standby should take over and service requests, both internal and external.

Power down the router for the internet router, utilize traceroute and ping to verify connectivity to google.com. Troubleshoot as necessary if testing fails. This might require tracking objects and/or utilizing IP SLA as well as a Layer 3 redunancy protocol such as VRRP/HSRP.

3. If the ASA at the active site fails, the ASA at the standby site should take over and service requests.

pull cables and/or power down the ASA, utilize traceroute and ping to verify connectivity to google.com. Troubleshoot as necessary if testing fails. Ensure the the standby ASA resumes the role of the active (By default it just uses a copy of the config of the active device).

4. If the ISA at the active site fails, the ISA at the standby should take over and service requests.

pull cables and/or power down the ISA, utilize traceroute and ping to verify connectivity to google.com; troublehsooting as necessary, I do not have any information/advice I can give on the ISA.

5. If the connection between the active and standby sites fails, branches should be able to communicate via the MPLS to the active site.

pull cables the MPLS Router, utilize traceroute and ping to verify connectivity to google.com and any IP addresses in the local and/or remote networks. Troubleshooting as neccessary,

6. If the MPLS router at the active site fails, the MPLS router at the standby site should be able to route traffic from the branches to the active site.

Power down the MPLS Router, utilize traceroute and ping to verify connectivity to google.com and any IP addresses in the local and/or remote networks. Troubleshoot as necessary if testing fails. This might require tracking objects and/or utilizing IP SLA with also utilizing VRRP/HSRP. Check the status of the standby group, check the track objects, etc.


Billy
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In the world of WAN, QoS is a pretty important topic for most, if not all, networks. Some WAN technologies have QoS mechanisms built in, but others, such as some L2 WAN's, don't have QoS control in the provider cloud.
PRTG Network Monitor lets you monitor your bandwidth usage, so you know who is using up your bandwidth, and what they're using it for.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question