Solved

server failure recovery

Posted on 2014-12-16
15
108 Views
Last Modified: 2015-01-02
I am trying to setup a server failure recovery strategy.

How does it sound?

1. I will use multiple A records pointing the same address to 2 different servers (see the discussion here: http://webmasters.stackexchange.com/questions/10927/using-multiple-a-records-for-my-domain-do-web-browsers-ever-try-more-than-one)
2. At normal conditions the server B forwards the traffic to server A programmatically to avoid any problem of multi A records
3. Server B checks if Server A runs good. If B sees that A is not running then stops forwarding the traffic.
4. Server B also starts triggering  a page at server A to stop it to avoid temporary recoveries (eg. every 5 seconds server B requests the page: IP_of_server_A/stoprunning.php)
5. Thanks to multi A records the browser will forward the requests to B when A is not running.
0
Comment
Question by:myyis
  • 6
  • 5
  • 2
  • +1
15 Comments
 
LVL 69

Expert Comment

by:Qlemo
ID: 40504415
Why do you want to prevent fallback to A? You are not concerned about sync, I suppose, as B isn't in sync anyway.
Other than that, it seems to be a working concept.
0
 

Author Comment

by:myyis
ID: 40504485
There is a continuous DB insert transactions. I will use one way sync. That means the A will write to B continuously. Therefore when A is down I want that no body inserts data. When A is online and stable again  I will write the new DB records to A.

How does that sound?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40521658
Where is the DB insert uniqueness handled? In PHP or in database?
0
Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

 

Author Comment

by:myyis
ID: 40521946
Sorry I don't understand what you mean DB insert uniqueness? Can you give an example?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40521961
How do you know that only one server can be up and running?
Normally you can have 2 or 200 inserting new records in database (how do you think facebook works)
0
 

Author Comment

by:myyis
ID: 40522039
1. In normal conditions 2 servers are up (multi A records),  all requests to backup server(B) are forwarded  to main server (A). So only A is receives requests.
2. Every insert query at A is also triggered at B
3. Always checks the inserted ID  after the transaction at B, it should be equal to the inserted ID of related transaction at A
 Therefore both servers always have the same records.

When server A  is down,
1.  it is made permanently down to avoid temporary recoveries (explained above)
2. Forwarding from B to A is stopped and the requests are accepted at B
0
 
LVL 14

Expert Comment

by:kenfcamp
ID: 40522202
What happens when server B goes down?
0
 

Author Comment

by:myyis
ID: 40522384
For DB consistency:
All the insert (also update and delete) queries generated at A  is written to a table. If they are successfully executed at B the related query is deleted from the table. If can not be successful that means that B is down, the row will not be deleted.  When B is recovered it will execute the queries that are at the table (at the queue).

At the user site: When B is down the browser will check for the working server (A) thanks to multi A records. (http://webmasters.stackexchange.com/questions/10927/using-multiple-a-records-for-my-domain-do-web-browsers-ever-try-more-than-one))
0
 
LVL 69

Assisted Solution

by:Qlemo
Qlemo earned 250 total points
ID: 40522412
Sounds good, but recovery from B to A needs some effort. But that could be eased if you use the same "logging table" approach on B.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40522616
Could it be mirrored database that fails over and two or 10 decoupled webservers approach highly available replication master of the moment ? Then you just need failover script for DB and wise config for webserver
0
 

Author Comment

by:myyis
ID: 40522671
Hi gheist,
If you're suggesting replication features (master-slave) of DB I can't do that because I am on shared environment and I don't have root access to the server.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40522702
I am just telling that you can use primitive DNS load balancing to balance between two apache app servers. There is no need to complicate the picture with kill scripts ensuring one application is running.

They both approach database you have at the same time

Problem solved, no radical customization. One server goes down, all user browsers handle failover until Monday you get back to work. Maybe add 3rd apacce server for christmas holiday, but "works for me" like this.
0
 

Author Comment

by:myyis
ID: 40522728
Hi Gheist
I have checked this link (https://www.digitalocean.com/community/tutorials/how-to-configure-dns-round-robin-load-balancing-for-high-availability)

At the end of the article it mentions that I have to have  DB replication. But I cannot have replication feature, that's why I am thinking a more complicated solution.
0
 
LVL 62

Accepted Solution

by:
gheist earned 250 total points
ID: 40522742
You just stated: There is no way you can make database highly available (in your position)

2 identical web servers will serve highly available web interface (if DB goes down both applications are down)
You can add some caching etc to survive DB restart, but that is all you can have.
For testing - set up both servers, just gracefully stop one apache and check that you can still browse your site.
rsync is your friend to synchronize content.
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

There’s a good reason for why it’s called a homepage – it closely resembles that of a physical house and the only real difference is that it’s online. Your website’s homepage is where people come to visit you. It’s the family room of your website wh…
Microsoft will be releasing the Windows 10 Creators Update in just a matter of weeks. Are you prepared? Follow these steps to ensure everything goes smoothly and you don't lose valuable data on your PC.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

761 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question