Monitoring and Validating Backups: A Part of Disaster Recovery Plan!

Data loss is something that an administrator fears. Being prepared with the knowledge and tools to work around troubles that can cause data loss is the right course of action against disaster and avoiding its consequences.

 
A redundant hardware and disk system helps, but surely the best hardware configuration comes with best investment. For designing a redundant hardware, following are some of the recommendations:

#1: Of course budget matters. Integrating hardware with multiple processors, network connections, and memory buses is important. In addition to this, the hardware should have potential of auto-correction and run successfully even after some basic errors.

#2: Support to hardware should be frequent. How quickly hardware components can be shipped if replacement due to damage would be needed or what is the possibility to store the components within site for quick fixing.

#3: No matter how much investment is being done on quality hardware, there is possibility that it can fail. Always monitor the performance of the hardware from time to time. Also, place Operating Systems on mirrored disk.

#4: For redundancy of the database and log files, RAID solution can be adopted so that the loss of a disk does not mean complete data loss. Or the database saved in SAN or NAS can be replicated to a different site. Now, this can also mean that DAG is set up with copies of the database and log files replicated at different locations.

Now, if the disaster has happened, how you react to it determines the downtime and productivity loss. With effective measures to react, the adverse consequences can be minimized. There are new and improved solutions offered in both Exchange 2007 and 2010 to deal with disasters by avoiding downtime.

Dial Tone Recovery
Following this strategy for recovery, an original database is dismounted from Server and a blank database is mounted on same Server so that users are able to send and receive emails through it while the damaged DB is recovered or restored. Once the database is recovered, it can be exported to respective mailbox.

Database Portability
The database from an Exchange Server can be mounted or exported to another Server of the same version and administrative group. If a Server fails or is not working well, but still access to database is available (example: through backup), the mailboxes can be moved to another Server. Further, PowerShell cmdlet “Set-Mailbox -database” can be run to direct user mailboxes to another Server.

Deleted Item Retention
If some items are deleted by the user, they are not permanently deleted. A copy is still stored in a mailbox on Server from where it can be restored under a specified time period. Also, a deleted mailbox remains available in disconnected state until retention period surpasses.

Setup/mode: RecoverServer
Configuration information about Exchange Server gets saved in Active Directory. In case of Server failure or crash, a new Server can be installed with same configuration settings by running Setup/mode: RecoverServer.

Understand Database Technology
The Exchange Server database is designed with an alert system. For example: If a page in the database file is corrupt and anything wrong happens to the disk, the online backup procedure fails with error 1018. For a failed database, different approaches can be made that include Disaster Recovery Analyzer, ESEutil, Isinteg etc. can be used.
 

Considerations to Configure Backups

The VSS technology introduced in Exchange 2003 has improved a lot till Exchange 2010. When snapshot of the database is taken for backup, all write operations on the DB as well as the log files are paused. This snapshot taking process takes a few seconds and after that all operations are resumed to normal.

Nevertheless, along with choosing a right backup maintenance technique and right software for backing up the database, there are a number of variables that should be taken into strict consideration. While the software takes a backup of the database, it should have less impact on users and mailbox accessibility. Following variables should be kept in mind while designing a backup:
 

• Consider the time required to backup data and the time needed to restore it. For this, discussion with right company executives should be done as to how long a down database is affordable: 60 minutes, 12 hours, or one day. Depending upon this factor, you have estimate as how much data can be restored at a given point of time after any disaster.
   
• Give careful thoughts to the fact as how much resources are required to backup and restore database. By default, Exchange does online maintenance of the database when the Server is idle (generally at non-working hours which is night) and checks it for any errors and fragmentation.  Know what that time is and stay alert as how backup would complete with the maintenance process if both of them happen to have occurred at the same time because their clash can cause load on the server. Keep a continuous watch on network bandwidth, Input/Output, and CPU load to find the best timestamp for backing up the database.
   
• It is important to define Recovery Point Objective. This will help to estimate the data loss that a company can tolerate along with the fact as how much data can be recovered. Different backup strategies have to be accepted for RTO where data loss of not more than one hour is accepted and where data loss of a day is accepted.
   
• The backup hardware used makes a difference as it can affect restoration process and consequently the Recovery Time Objective (RTO). Choose an optical or disk based backup technology that helps in backup restoration in comparatively less time.

 
After considering all these facts, a suitable backup plan (Full, Copy, Increamental, or Differential) backup can be created as a part of DR strategy.




 

How Important it is to Test the Backup?

 
Some years ago, I worked with an institute where a full backup of all students and teachers was maintained every night.   One day, one of the drives failed and backup was considered as the solution to recover data. Till the time failure happened, the backups were not tested.
 
When the backup was restored, it showed up that the media where backup was stored is worn out. The backup was completely useless and nothing could be restored from it.
 
The backup media were sent to a data recovery company where special tools in a  clean room were used to retrieve data from the media. It took 4 days to recover the data and of course a lump-sump too.
 
If there is complete reliance upon backups, then it is important they are tested for successful restoration or any harm to the media. Only a complete and valid backup can help to work around disaster at an estimated time and cost.
 



 
However, using command-line and GUI solutions for recovery and repair of database can be a great help. Understanding the right action against the database: Repair or Recover is important in order to get back database back within the defined RTO with the help of Microsoft’ native solutions.

Know about the Right Course of Action in Exchange Server Disaster Scenario: Repair or Recovery?  In the Upcomming Artcle - Recovery Tools for Exchange Database : Options and Operability