Backup HyperV VM is causing it to go in saved state
I have Symantec Backup Exec 2010R3 and the HyperV agent which I have been using for over a year to back up my HyperV VM's every night. It has worked flawless for this time, until a week ago. Now, what is happening is when it hits just one particular VM, the VM will go into a saved state while the backup starts. It will then reconnect and start back up, but I do lose connectivity to it for about 2-3 minutes while it does that. This has never happened before. I am running 2008R2 SP1 Enterprise with the VM's in a failover cluster with cluster shared volumes iscsi to my SAN. I have tried moving the CSV to other hosts but that has not made any difference. The VM still backs up entirely as I would expect, but Im at a loss for what would all of a sudden make it start going into a saved state. There are several other VM's on the same host server which do not have the same symptoms.
Mike
Have you installed anything new on that particular VM? Almost sounds like it goes into a save state to backup certain files (almost like they are in use and locked).
ASKER
No, nothing that comes to mind. Its a SQL server so its pretty much a hands-off server as things getting installed or changed. But the backup is happening at the cluster level, so its not much of the OS as far as I can tell.
Thanks.
Thanks.
Do you have the hyper-v integration services installed on the VM?
Your VSS writers could be failing inside the VM. check the status of your VSS writers (can do a vssadmin list writers). Microsoft allows for two types of backups, either it tosses it into "saved state" by default, or uses the VM's VSS Writers.
http://msdn.microsoft.com/en-us/library/dd405549(VS.85).aspx
I suggest removing and reinstalling the agent if something is wrong there.
Your VSS writers could be failing inside the VM. check the status of your VSS writers (can do a vssadmin list writers). Microsoft allows for two types of backups, either it tosses it into "saved state" by default, or uses the VM's VSS Writers.
http://msdn.microsoft.com/en-us/library/dd405549(VS.85).aspx
I suggest removing and reinstalling the agent if something is wrong there.
ASKER
Thanks, I will try the agent. None of the writers show errors. Yes, I do have the integration services installed, and since this has been running perfectly for a year, I was at a loss for what could have changed since I havent made any changes. But a reinstall of the agent is something I havent tried.
Reinstall the Integration Services.
What is the OS of the VM?
What is the OS of the VM?
If a VM go to saved state at the beginning of its backup, its means - for your hyperV's point of view- that your VM could'nt be backuped ONLINE.
To be backuped online, a WINDOWS VM needs to have HyperV Integration Services in it (in order to hyperV host could query its state and its config) AND a VSS layer that works correctly inside the VM.
Rem : any Linux VM will be backuped offline (in saved state) even with integration services, because of the lack of a VSS (snapshot) layer (or similar) in the Linux OS?
So:
- check integration services in the VM
- check also VSS layer in the VM (system and application eventlogs)
To be backuped online, a WINDOWS VM needs to have HyperV Integration Services in it (in order to hyperV host could query its state and its config) AND a VSS layer that works correctly inside the VM.
Rem : any Linux VM will be backuped offline (in saved state) even with integration services, because of the lack of a VSS (snapshot) layer (or similar) in the Linux OS?
So:
- check integration services in the VM
- check also VSS layer in the VM (system and application eventlogs)
ASKER
Thanks for the replies. The VM is a 2008R2 server. I will reinstall the integration services this evening after hours and see if that corrects it.
ASKER
I have reinstalled the integration services, which of course cannot be uninstalled as its a default install on 2008R2, but only lets me repair, which I have done twice now. I have also uninstalled and reinstalled the backup exec agents both on the host server and on the VM itself, just to be thorough. No go... still saves the state.
So I will next be trying to delete the VM (not deleting the VHD's of course... ) and recreate. Just for a test.
So I will next be trying to delete the VM (not deleting the VHD's of course... ) and recreate. Just for a test.
sometimes, you can just do an insert integration services disk, on the VM, if you connect to it, if you install it then off the optical drive in the VM itself, it will reinstall. I know doing this fixed this very issue for me, so maybe it's something you can still try
ASKER
None of these solutions have helped, unfortunately. I have simply excluded that particular VM from all the others for my nightly backups as it was beginning to seem like it was corrupting some data in the SQL database that resided on that VM when it was placing the VM in a saved state requiring a re-index. Either way, I will just close the question if any other new ideas are not found. Thanks for all who have suggested help.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I found the solution on my own