The purpose of this article is to document the steps needed in order to restore an Exchange 2003 Information Store after you have migrated to Exchange 2007 and prior to the decommissioning of your previous Exchange 2003 organization. I will not go into a step by step procedure for every process along the way, but will provide the necessary details and concepts that I feel are key to making this work.
Disclaimer: The steps outlined herein are to be used at the readers discretion and the author will take no responsibility in any damage caused to any network as the result of following this procedure. Green Horn Admins proceed with caution!
In this day and age, it's quite common for an Exchange Admin to receive a request from the Legal Department to pull a mailbox or group of mailboxes back from tape that have been targeted for legal discovery. More often than not, these tapes are typically many months or years old. But how is this restore possible after you have already migrated all your users over to an Exchange 2007 Server? You certainly don't want to keep your old Exchange Server hanging around in your production environment just for this purpose! To add to this nightmarish thought, Exchange 2007 doesn't support the recovery of an Exchange 2003 Information Store because the whole database format has been changed.
This is the question I asked myself recently during my own 2003/2007 migration. I had to ensure that this process would be possible, knowing full well that I would find myself in this situation sometime in the future.
For me, I was fortunate enough to have a VMWare ESX Cluster sitting around that had enough storage and resources available to accommodate part of my old Exchange environment. Early on in this process, I wasn't exactly sure how I would go about making this happen, so I considered this a bit of an experiment that I would work on as time allowed since I didn't have any lawyers beating down my door just yet.
A few things I knew I would have to address right away were acquiring a copy of an active Domain Controller, an active Backup media server and of course a copy of my co-existing Exchange 2003 server that wasn't yet decommissioned. In my environment I chose to utilize the Physical to Virtual (P to V) capabilities of VMWare to create these server copies. Once the P to V process on each of these servers was complete, they would be immediately banished to their own dedicated, non-uplinked virtual switch inside VMWare never to see the light of day again.
As a point of note, it is critical this segmented environment is never connected to your production network. If it is allowed to talk to the other servers, it could cause major conflicts.
A final note before getting too far along is to mention that these steps may seem like a lot of work at first, but once the initial setup is done, it actually doesn't turn out too bad in terms of the amount of time it takes. I can now complete a restore from tape to the final product of a burned CD with targeted .PST files in less than an hour. Not too bad in my opinion!
P to V the Domain Controller:
The first server that was targeted for the P to V process was one of my Domain Controllers (DC). Ideally the one I would choose will hold the majority of the FSMO Roles to reduce the amount of Role Seizing that needs to occur later on. The previous statement is of course assuming you have more than one DC in your environment. In most circumstances, I would say to just build a fresh virtual machine DC and not bother with the P to V headaches. However, in this case, I wouldn't recommend adding a new DC object to your production Active Directory Schema, since it will be permanently sitting off the grid when you're done.
I was fortunate enough to not run into any major issues with the AD configuration on the segmented DC after the P to V process. There was a bit of unexpected DNS clean up that needed to occur upon first boot up of the segmented DC since the Virtual NIC didnt retain the Physical NIC IP configuration. After a few reboots and the installation of the VMWare Tools was completed, everything was happy for the most part. Once the NIC settings were happy, I then proceeded to seize any FSMO roles held by the other DCs using NTDSUtil; this would allow me to rebuild a functional Active Directory environment, segmented from the production network. After all FSMO roles were seized, I then proceeded to manually remove the references to the other DCs that would not be present inside my segmented AD environment. The goal here was to keep AD as happy as possible to support long term usage without needing to worry about Event Log storms related to AD replication and tombstoned objects that will never be present again past the 60 day limit marker. It's also a good idea to go through the gamut of AD tests to ensure everything is working. For example, you can test with DCDiag, pings, NSlookups and so on.
Once you have a good handle on the Domain Controller and AD related cleanup tasks, you can feel pretty confident about moving on to the next server targeted for the P to V process.
P to V the Exchange 2003 Server:
In my experience, the key to a successful P to V migration of an Exchange Server is to ensure that ALL Exchange-related services are not running during the process. For me, I stopped them and then changed their startup types to Manual. Before kicking off the P to V import, it's a good idea to check Windows Task Manager to confirm there are no Exchange related applications still running. One thing to also think about prior to importing your Exchange Server is whether or not you actually need the current Exchange database files (EDB/STM) inside the segmented environment. In my case I did not need them, so I created the virtual disks to reflect the total size of the drive/partition minus the size of the EDB & STM files. If you have limited SAN space in your virtual environment and don't want to wait for the migration of hundreds of gigabytes of Exchange database files, it's worth the effort to move or delete these files prior to importing if possible. If you can't delete or move these files prior to import, the other option would be to just import the C: drive of the Exchange Server. Then you can manually create the other needed virtual disks prior to boot up of the now migrated virtual machine. Once booted, you can then configure the new, smaller disks inside Windows Disk Management to reflect the same drive letters used on the physical server.
Keep in mind during this process that the Virtual Exchange Server will not be happy that some of the drives are missing certain data and certain drive letters have changed. However, the Exchange Services will still be set to Manual, so this won't be a problem the moment you boot the new, virtualised server. If you do use this method, you will still need to find a way to get the program file data not imported earlier from the physical Exchange server to the segmented virtual Exchange server. One easy method is to burn those files to a DVD and then attach the DVD to the virtual machine. Once attached you can then copy the needed program files onto the newly created smaller virtual disks. Once you are done, the setup should reflect the physical server's configuration minus the Exchange EDB & STM files.
Next up would be to change the start up type of the Exchange Services back to Automatic in preparation for the first clean boot up. After the Services have been changed and prior to your first clean boot up, you will have to also re-configure the virtual NIC as you did on the DC earlier.
One final note on this process: after you clean boot the virtual machine and confirm all the services are running, it's a good idea to go through the steps of changing all of the Exchange roles to point to your single instance Exchange server. This of course assumes you had multiple Exchange servers on your previous production network. Some steps to note here are making sure the GAL, the Free Busy Info and the Routing Group Master server settings are all located and owned by the segmented Exchange Server in the virtual environment. Again, the goal here is to have a happy, mostly functional Exchange server dedicated to Recovery Storage Groups that can be sustained for the long term without worry of Event Log floods and other headache inducing error messages.
P to V the Backup Media Server:
Similarly to the Exchange Server, it's a good idea to stop and change the startup type of all backup related services to manual prior to P to V import. In my case, I was using Symantec Backup Exec 11d on my media server. The steps to P to V your backup server are pretty much the same as your Exchange Server and you can take the same drive space considerations into account again. You could choose to only import the C:\ drive first. Some backup servers have massive catalog file drives that may or may not be needed in your segmented environment. If you don't need all the extra files, just create a virtual disk the size of the program files and exclude the catalog files. Again, just create a DVD with the program files and copy them to your new, smaller virtual disks. Reconfigure the virtual NIC again and work towards your first clean boot of the virtual backup media server as in the previous steps.
The Interesting Part:
Things start to get interesting from this point forward now that you have imported the necessary servers and worked out any networking and services bugs. At this point you should have a functional DC that can fully communicate with the other servers in the segmented environment and all servers in the virtual environment should be able to fully communicate with each other.
The next big challenge you are faced with is how to get the data located on the tape media into your virtual environment. Since most of us probably don't want to deal with the nightmares of connecting a tape drive to your ESX servers, the alternative is to perform what is known as a media migration. The process involves copying the data from a backup tape to a backup to disk configuration. The steps in this article are centered around using Backup Exec (11d) in my case, but may also be done using other backup software assuming similar feature sets are available. Another assumption being made here is that there is enough disk storage on the media server available to accommodate the size of the tape media being migrated.
The first thing to do at this point is to insert the tape into your tape library that has the data you need to restore. Then, create a Backup to Disk virtual device within your backup software and point the storage location to a local drive that has enough free space available. At this point, you should perform an inventory of the media bays in the tape library. Once the inventory has completed successfully, the next step would be to catalog the media tape of interest.
One thing I found helpful to speed up the process was to temporarily rename my default backup catalogs folder prior to executing the catalog media command on the tape of interest. The reason for this in my case was related to having several hundred gigabytes of old catalog files in the default directory from previous backups. When I didn't rename this directory beforehand, the backup software would attempt to catalog everything inside this folder, which took way too long. If you rename the folder first and then restart all the Backup Exec Services, it will
recreate a blank catalogs folder for you. When you now catalog the tape of interest, it will be significantly faster and the folder will only contain the necessary catalog files for the task at hand.
After the restore is successful, you can rename your old catalog folder back to its original name, and restart the Backup Exec Services again to get back to normal.
Once you have cataloged your tape of interest the next step is to perform the media migration. In Backup Exec 11d, there is an option located in the left-hand pane on the Job Setup tab called New Job To Duplicate Backup Sets. After selecting this, you want to keep the default Duplicate Existing Backup Sets option and click OK. Now you will be presented with the familiar job setup window. You should choose the View By Resource tab and then you can browse the Exchange Server resource containing your data. Once you choose the Information Store of interest, the next step to is change the destination of this restore to point to the Backup To Disk device created earlier in this process. I didn't need to mess with any other options inside this window so I proceeded to run the job.
Once the job fires off, you should soon start to see 2GB .bak files populating your Backup To Disk location. If everything worked as planned, the job will finish and you should be looking at a disk-based version of your backup tape that only contains the Information stores you chose during the job setup.
Now that we have the necessary data located on a friendlier media format, it's not too difficult to move this data into our segmented virtual environment. The way I moved the data into the VM environment without exposing the segmented machines to the production network was through the use of a fourth standalone Windows XP client that I could attach and detach between the segmented and production networks. This VM client was created with a fairly large C: drive large enough to hold the .bak files from the media migration process. It was not joined to the production domain for obvious reasons. I would fire up the standalone client attached to the production network and then open a session to it through the Start > Run command line calling it via \\Ip Address\c$. You can then login using the local Administrator account. This gave me access to the disk where I could then proceed to copy all the .bak files to.
So we now have the .bak files located on the standalone VM client. The next step would be to shut down the client. At this point it's time to also shutdown the virtual Backup Media server located in the segmented environment. Now that both machines are shutdown you will want to Edit Settings on the Backup Media server VM. The goal is to add a new disk to the Backup Media server by using the disk of the standalone Windows XP machine. This is done by adding a new, existing disk to the VM and then browsing to the location of the standalone XP machine's virtual disk and adding it to the Backup Server.
Now that the disk is attached to the Backup Server, you can power it on again. You will probably see a message come up about the newly attached disk during boot up, but this can be ignored (click OK on the default option selected).
Once the Backup Server is booted, you can login and see the new disk that contains the .bak files you migrated earlier. The hard part of getting the original tape data into your VM environment without needing to attach a tape device to your ESX server is now complete.
Preparing the migrated media on the virtual Backup Server:
At this point the next steps are to create another Backup To Disk Device but this time on your virtual Backup server. What you want to do is point the Backup To Disk location to the folder you are storing the .bak files copied from earlier. Once this step is complete you will then want to Inventory and Catalog this location. When you're done, you will end up with a restorable, disk based version of your original physical tape media that is now located inside your virtual segmented environment.
One additional preparation step needed on the virtual Backup server involves editing the Registry to avoid issues while attempting to redirect your restore to the exchange server. I found this step necessary during my attempts because the Exchange Server I was using didn't originally own the database I was attempting to restore, but it did belong to the same Exchange 2003 Organization of my segmented Exchange 2003 machine. Without this registry modification, the restore attempt would fail, because Backup Exec would not be able to find the original Exchange Server.
Open Regedit and browse to HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\ESE.
Create a New DWord key with the name of ignore mount state and set its value to 1
Close Registry Editor
Preparing the virtual Exchange Server:
On your virtual Exchange Server, you will need to perform the same registry change listed above. The next step would be to create a new Recovery Storage Group using the exact same name of the original Storage Group holding the database you are trying to recover. Once the RSG is created, you are ready to add the database needing recovery. Also make sure that the database created within the RSG is marked that it can be overwritten by a restore.
A word of caution in regards to the registry change performed earlier on both servers: unless your intention with this environment going forward is strictly for the purposes outlined here in the article, I would recommend deleting the registry change after you are finished with the restore process. This registry key can be very dangerous as it removes the built-in protections of the software allowing for blind overwriting of a mail store.
Restoring the Exchange 2003 Data:
We are now ready to perform a restore from our backup server. The process that follows is pretty cut and dry for a redirected restore of Exchange except for a few additional options I recommend considering. First you will want to make sure that the device being used during the restore is your Backup To Disk device created earlier. The next setting to configure is the Destination of the restore - this should be the Microsoft Exchange Redirection option. Check off the Redirect Exchange Sets box and type in the Exchange Server name you are using in the restore. Next, configure the Settings section of the restore job. Select Microsoft Exchange and then under the Exchange 2000 and Later section, choose the option to Purge Existing Data and restore only the databases and transaction logs from the backup sets. Lastly, you will want to uncheck Mount Database After Restore.
We are now ready to restore the Exchange 2003 database to our Recovery Storage Group on the virtual Exchange 2003 server. Start the job. If everything is set properly, you should see the job complete pretty quickly.
Retrieving your restored mail:
Once the restore job is complete, you can then mount the database within the Recovery Storage Group manually and take a look at the mailboxes inside. After I saw the mailboxes which were available from the restore process, I dismounted the database and used a third party extraction tool to pull the individual mailboxes out to PST files. One such utility is called Kernel For Exchange which is available online for reasonable cost. The software allows you to point to an EDB file and browse and extract individual mailboxes to a PST. You could also try to use the free Exmerge utility, but in my case it wouldn't work because it was telling me the mailboxes had already been moved.
Once you've completed the extraction to a PST, all you need to do at this point would be to get the data back out to your production environment through the same methods you used to get it from your production environment. Use the Windows XP standalone machine again as your transfer method and you are all set.