Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

server failure recovery

Posted on 2014-12-16
15
107 Views
Last Modified: 2015-01-02
I am trying to setup a server failure recovery strategy.

How does it sound?

1. I will use multiple A records pointing the same address to 2 different servers (see the discussion here: http://webmasters.stackexchange.com/questions/10927/using-multiple-a-records-for-my-domain-do-web-browsers-ever-try-more-than-one)
2. At normal conditions the server B forwards the traffic to server A programmatically to avoid any problem of multi A records
3. Server B checks if Server A runs good. If B sees that A is not running then stops forwarding the traffic.
4. Server B also starts triggering  a page at server A to stop it to avoid temporary recoveries (eg. every 5 seconds server B requests the page: IP_of_server_A/stoprunning.php)
5. Thanks to multi A records the browser will forward the requests to B when A is not running.
0
Comment
Question by:myyis
  • 6
  • 5
  • 2
  • +1
15 Comments
 
LVL 69

Expert Comment

by:Qlemo
ID: 40504415
Why do you want to prevent fallback to A? You are not concerned about sync, I suppose, as B isn't in sync anyway.
Other than that, it seems to be a working concept.
0
 

Author Comment

by:myyis
ID: 40504485
There is a continuous DB insert transactions. I will use one way sync. That means the A will write to B continuously. Therefore when A is down I want that no body inserts data. When A is online and stable again  I will write the new DB records to A.

How does that sound?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40521658
Where is the DB insert uniqueness handled? In PHP or in database?
0
How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

 

Author Comment

by:myyis
ID: 40521946
Sorry I don't understand what you mean DB insert uniqueness? Can you give an example?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40521961
How do you know that only one server can be up and running?
Normally you can have 2 or 200 inserting new records in database (how do you think facebook works)
0
 

Author Comment

by:myyis
ID: 40522039
1. In normal conditions 2 servers are up (multi A records),  all requests to backup server(B) are forwarded  to main server (A). So only A is receives requests.
2. Every insert query at A is also triggered at B
3. Always checks the inserted ID  after the transaction at B, it should be equal to the inserted ID of related transaction at A
 Therefore both servers always have the same records.

When server A  is down,
1.  it is made permanently down to avoid temporary recoveries (explained above)
2. Forwarding from B to A is stopped and the requests are accepted at B
0
 
LVL 14

Expert Comment

by:kenfcamp
ID: 40522202
What happens when server B goes down?
0
 

Author Comment

by:myyis
ID: 40522384
For DB consistency:
All the insert (also update and delete) queries generated at A  is written to a table. If they are successfully executed at B the related query is deleted from the table. If can not be successful that means that B is down, the row will not be deleted.  When B is recovered it will execute the queries that are at the table (at the queue).

At the user site: When B is down the browser will check for the working server (A) thanks to multi A records. (http://webmasters.stackexchange.com/questions/10927/using-multiple-a-records-for-my-domain-do-web-browsers-ever-try-more-than-one))
0
 
LVL 69

Assisted Solution

by:Qlemo
Qlemo earned 250 total points
ID: 40522412
Sounds good, but recovery from B to A needs some effort. But that could be eased if you use the same "logging table" approach on B.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40522616
Could it be mirrored database that fails over and two or 10 decoupled webservers approach highly available replication master of the moment ? Then you just need failover script for DB and wise config for webserver
0
 

Author Comment

by:myyis
ID: 40522671
Hi gheist,
If you're suggesting replication features (master-slave) of DB I can't do that because I am on shared environment and I don't have root access to the server.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40522702
I am just telling that you can use primitive DNS load balancing to balance between two apache app servers. There is no need to complicate the picture with kill scripts ensuring one application is running.

They both approach database you have at the same time

Problem solved, no radical customization. One server goes down, all user browsers handle failover until Monday you get back to work. Maybe add 3rd apacce server for christmas holiday, but "works for me" like this.
0
 

Author Comment

by:myyis
ID: 40522728
Hi Gheist
I have checked this link (https://www.digitalocean.com/community/tutorials/how-to-configure-dns-round-robin-load-balancing-for-high-availability)

At the end of the article it mentions that I have to have  DB replication. But I cannot have replication feature, that's why I am thinking a more complicated solution.
0
 
LVL 62

Accepted Solution

by:
gheist earned 250 total points
ID: 40522742
You just stated: There is no way you can make database highly available (in your position)

2 identical web servers will serve highly available web interface (if DB goes down both applications are down)
You can add some caching etc to survive DB restart, but that is all you can have.
For testing - set up both servers, just gracefully stop one apache and check that you can still browse your site.
rsync is your friend to synchronize content.
0

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Get to know the ins and outs of building a web-based ERP system for your enterprise. Development timeline, technology, and costs outlined.
As technology users and professionals, we’re always learning. Our universal interest in advancing our knowledge of the trade is unmatched by most industries. It’s a curiosity that makes sense, given the climate of change. Within that, there lies a…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question