Azure Backup Failure After In-Place Upgrade

After performing an in-place upgrade of an Azure VM from Windows Server 2012 to Server 2016, I found that the scheduled nightly backup process failed with the following error:

• Error Code - GuestAgentPluginProcessingError
• Error Code - VM agent failed in processing Extension command.
• Recommended action - Please restart the VM Agent service

Restarting the VM Agent Service didn't help, nor did restarting the server itself. Multiple attempts to manually run the backup would still cause it to fail after about 50 minutes. Checking Event Viewer turned up nothing useful and there were no errors logged for Azure Backup or anything related.

After consulting with Microsoft support, I learned that you can look under c:\Packages\Plugins\Microsoft.Azure.RecoveryServices.VMSnapshot\x.x.x.x\Status and check the latest log to see if there is a problem, and I discovered this:

To fix this, we had to force reset the Microsoft.Azure.RecoveryServices.VMSnapshot module. Simply reinstalling the guest agent wasn't enough to rebuild it. To make this work, the following steps were taken:

1. Stop the service named Windows Azure Guest Agent
2. Rename the folder C:\Packages\Plugins\Microsoft.Azure.RecoveryServices.VMSnapshot
3. Delete the registry key at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure\HandlerState\Microsoft.Azure.RecoveryServices.VMSnapshot_XXXX
4. Start the service Windows Azure Guest Agent
5. Trigger a manual backup from Azure portal and wait for it to complete.

After doing these steps, the Azure backup successfully completed.

Screenshot from step 3:

When you restart the service, you'll see the new Microsoft.Azure.RecoverServices.VMSnapshot folder appear: