Link to home
Start Free TrialLog in
Avatar of Aaron Moukperian
Aaron Moukperian

asked on

Active Directory repair after roll back. Hyper-V virtual machine

Recently rolled back our AD server (Windows Server 2008) due to a ransomware issue. After doing an instant recovery (Veeam backup & Replication) of the VM, the active directory is now corrupt. Is there a way to repair it?
Avatar of Daryl Bamforth
Daryl Bamforth
Flag of United Kingdom of Great Britain and Northern Ireland image

Not really, AD does not work well with snapshots,

Have a look at this Microsoft Article for considerations when running DC's in Hyper-V.

https://technet.microsoft.com/en-us/library/virtual_active_directory_domain_controller_virtualization_hyperv(WS.10).aspx

Specifically the back-up sections.

Do you have any other back-ups?
How big is your environment? (How many users/objects?)
Avatar of M A
The only recommended way is recover backup.
Do you have another backup?
Do you have  ADC?
Avatar of Aaron Moukperian
Aaron Moukperian

ASKER

I only have the other roll back dates within Veeam, no other backup in place.  We only have the one DC running AD, there is another VM running the exchange server. It's not a large environment only 40 AD users
You can try with another date backup restore from Veeam but not 100% sure it will work.
Probably quicker to start from scratch.

Look though the microsoft link,  especially at the backups sections.
What is the error u get when u try to login after restore backup?
Can you see "Applying computer settings"?
I can get into the server but when I go into active directory and I access a user when I try and to view the member of tab i get operation error has occurred. Also, users are unable to login to the domain.
Have the AD services been started back up?
The Active Directory Domain services and web services both show started
There is only the single DC
I also get an error on the Exchange VM when trying to access the exchange management console "Kerberos" authentication failed: connecting to the remote server failed with the following error message: The WinRM client received an HTTP server error status (500) the remote service did not include any other information about the cause of failure. This started after the rollback.
The Exchange error is because Exchange was linked into your Active Directory. With this gone it cannot authenticate any more.
I can log them in with the administrator account on the domain just not their own login if that matters
which Administrator account? The Domain Admin one? Or machine local?
The domain admin is working on their machines
Can you get into the AD management console?
I can access the AD console and I can see all the users
What is the date/time of the AD? if you rolled it back is it wrong?

Try re-adding one of your PCs back into the domain (reset the pc object), reset a users password and see if they can now log in.

Restoring from backup may have broken the trust relationship between computer objects and the domain.
I checked the date and time it is correct on all servers. I'm attempted to reconnect a PC now
Looks like it can join the domain without error but when I go to log in, says incorrect password. Attempting to reset the password from AD gives An internal error has occurred
Is it not possible to repair AD using NTDSUTIL in active directory repair mode?
do you not have any backups that you can restore which work ?

How many user and computers ?
I have other rollback dates from earlier in the week. Should I try one of those? I used the one from Thursday (the issue occurred Friday)
ASKER CERTIFIED SOLUTION
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Agree with Lee.
You should test your backup, And Veeam allows to restore backup without effecting the production network.
Please try to restore another backup and update the status here.

Meanwhile create 2 new VMs and start installation of new domain controller and new exchange server.
Start copying the ED file from Exchange server to this server for mounting the DB.
In case if that restore fails you can continue creating these accounts and install Exchange and mount the database.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I tried running a repair on the database but to no luck. It's a small environment so I'll end up rebuilding it. I wasn't the one that setup this environment to begin with but I appreciate the tips.
At least you learned something and hope if your AD is compromised in the future you have a solid DR plan.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
So since my current AD is corrupt I can't join a secondary DC to the existing. Is my only option to backup the data from the main DC format and reinstall?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
The secondary DC would have only been a viable if it was already joined prior to the corruption.

The damage has been done and you do not have a true backup of the system state so there is no way to do restore (non authoritative restore).
That's correct I don't have the Windows system backup. So I need to take the current corrupted Active Directory VM offline and bring up a new VM and start fresh?
Yes, that's right.

Just disconnect it's NIC so you can still have the console open for it when you are creating users/groups/devices etc...
And rejoin all your workstations and servers.
Since the domain is on that server do I need to change the domain on the old server from or I simply disabling the nic will work?
If you have an additional DC (ADC) you could seize the existing DC and make your systems up.
You should have working backup offsite. Test the backup and copy that backup to a NAS or HDD and save/keep it offsite on a monthly basis.
If you have budget please try to have a DR site.

If I am not mistaken there is not primary DC and secondary DC concept after 2008 server. It is replaced with DC and ADC.
FSMO holder is considered as DC. Please correct me if I am wrong.
You are technically creating a completely new domain.
Your old dc should not even exist anymore.
As far as your new install is concerned it is the Primary DC as the domain only exists on the old, corrupted VM. So as long as that one is off the network (disconnected NIC does this nicely) the new VM will know nothing about it and be quite happy to install away and be authoritative for your domain.
Thanks, I'm setting the new up the new VM now
You have some work ahead of you.  Good luck and stay calm.
Also remember to use a backup tool that is a true system backup and test it regularly.
BTW If you do not have your Exchange database(EDB file) please export emails to PST on all your outlooks.
Once PC is disjoined from old domain you lost access to the previous outlook profile.
http://support.sherweb.com/Faqs/Show/how-to-export-an-outlook-2010-pst-file-exchange-2010
https://support.office.com/en-us/article/Export-or-backup-email-contacts-and-calendar-to-an-Outlook-pst-file-14252b52-3075-4e9b-be4e-ff9ef1068f91
Thanks everyone, for your input on this issue. I've since setup the new VM server with AD. Question regarding the old VM sever with AD, it is possible to just remove AD and have it act as a file server or will that potentially mess with the new AD Server?
You could try and remove AD role which will uninstall all the AD components and return this server to a server in a workgroup and then you can add it to your new domain
Should be fine as long as the demotion is clean. You will need to rejoin the machine as a member server.

There will still need to apply new ACL since the AD is new.

Also best practice is not to have your file server live on your DC.
The last issue I'm having is being able to reconnect to the exchange server since the corrupt Active Directory. We're moving to office 365 but need to access the exchange server to migrate the old email. Do I need to connect the exchange server to the new AD?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
You mentioned that have instant recovered  domain controller using Veeam. So it looks like you have valid backups of domain controller. If your backup is properly configured (application aware processing was enabled within dc backup job settings) you should be able to do authoritative restore of this domain controller. So if you have backup prior to date when rollback occure it should work for you. Pls take a look here on how to do authoritative restore of dc:
https://www.veeam.com/kb2119
Unfortunately I'm not exchange expert so others needs to point you what to do with exchange server after authoritave restore of dc. And/or if it this is supported in such configuration. So it should be possible depending on jour job config.
No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Split:
-- Andrew Hancock (VMware vExpert / EE MVE^2) (https:#a42347055)
-- MAS (https:#a42347426)
-- Daryl Bamforth (https:#a42347198)
-- Andrew Hancock (VMware vExpert / EE MVE^2) (https:#a42347127)
-- yo_bee (https:#a42347118)
-- Lee W MVP (https:#a42347059)


If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

Pber
Experts-Exchange Cleanup Volunteer