AD USN rollback error
I have a problem whereby the two domain controllers are out of sync and no longer replicating. It is a VM environment and the problem occurred when I created a copy of DC1 (server 2003) as a means to increase the disk size. The new VM was not successful so I took it offline and restarted the original. However, this created a problem with the second DC (server 2008) as the active directory was now out of sync between them.
DC1 (2003) is our original server that was migrated physical to virtual and hosts the DHCP, DNS and shared files (it was our single server). DC2 (2008) was created as a second domain controller for availability. DC2 holds all of the FSMO roles.
I tried demoting DC1 (after moving DFNS and DHCP to DC2) but this also removed it from the domain. Despite cleaning he metadata on DC2 I was not able to re-join DC1 back on to the domain so I reverted to snapshots to start again.
What I would like to do is remove DC2 and have DC1 be the single domain controller, then I can be sure that all of the DC1 services will remain intact. When this is done I can then create a new second domain controller that takes the AD version from DC1. I think that I just need to seize the FSMO roles on DC1 and demote DC2, but need to know that DC1 will just start fully functioning as a single domain controller using its version of AD. Advice appreciated please.
Thanks.
DC1 (2003) is our original server that was migrated physical to virtual and hosts the DHCP, DNS and shared files (it was our single server). DC2 (2008) was created as a second domain controller for availability. DC2 holds all of the FSMO roles.
I tried demoting DC1 (after moving DFNS and DHCP to DC2) but this also removed it from the domain. Despite cleaning he metadata on DC2 I was not able to re-join DC1 back on to the domain so I reverted to snapshots to start again.
What I would like to do is remove DC2 and have DC1 be the single domain controller, then I can be sure that all of the DC1 services will remain intact. When this is done I can then create a new second domain controller that takes the AD version from DC1. I think that I just need to seize the FSMO roles on DC1 and demote DC2, but need to know that DC1 will just start fully functioning as a single domain controller using its version of AD. Advice appreciated please.
Thanks.
As you mentioned on DC2 you have performed metadata cleanup to remove DC1.Againg you restore the snapshot of DC1.Correct me if I am wrong,
DC1 will not sync with DC2 as DC2 does not have instances of DC1.I will recommed to create new VM and promote the server back as DC instead of demoting the existing DC1.
As DC2 health is good and is also acting as FSMO role holder server are any client facing any issue.You can test the same but ensure that primary dns setting of clients is pointing to DC2.Shutdown the restored DC1 VM and then perfrom the test.
Also ensure on DC2 instances of DC1 is removed if present.
http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Note:Configuring DC either from clone/snapshot/image is not recommended.USN Rollback occurs when an Active Directory Domain Controller is restored via a snapshot or imaging process. Microsoft considers this a non-supported method of restoring Active Directory and it is this type of method that causes an Update Sequence Number (USN) rollback, because it results in the USN on the restored DC to be lower than what the other Domain Controllers are using.
To confirm if the server is in usnrollback check the below parameters.
*Netlogon service is in paused state.
*Event id 2103 will be logged whic will state that The Active Directory database has been restored using an unsupported restoration procedure.
*DSA Not Writable key with value 4 will be created in HKLM\System\CurrentControl
If above is true then to fix the issue you need to demote/promote the DC.You cannot demote the faulty DC gracefully you need to do forcefull removal.You need to ran dcpromo/force removal and then run matadata cleanup on other DC(healthy) to remove the instance of faulty DC from AD database and DNS.
Once done you can promote the Server back as DC.If faulty DC is FSMO role holder you need to seize the FSMO on other DC.
Reference link
Forcefull removal of DC: http://support.microsoft.com/kb/332199
Metadata cleanup: http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Seize FSMO role: http://www.petri.co.il/seizing_fsmo_roles.htm
Hope this helps
DC1 will not sync with DC2 as DC2 does not have instances of DC1.I will recommed to create new VM and promote the server back as DC instead of demoting the existing DC1.
As DC2 health is good and is also acting as FSMO role holder server are any client facing any issue.You can test the same but ensure that primary dns setting of clients is pointing to DC2.Shutdown the restored DC1 VM and then perfrom the test.
Also ensure on DC2 instances of DC1 is removed if present.
http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Note:Configuring DC either from clone/snapshot/image is not recommended.USN Rollback occurs when an Active Directory Domain Controller is restored via a snapshot or imaging process. Microsoft considers this a non-supported method of restoring Active Directory and it is this type of method that causes an Update Sequence Number (USN) rollback, because it results in the USN on the restored DC to be lower than what the other Domain Controllers are using.
To confirm if the server is in usnrollback check the below parameters.
*Netlogon service is in paused state.
*Event id 2103 will be logged whic will state that The Active Directory database has been restored using an unsupported restoration procedure.
*DSA Not Writable key with value 4 will be created in HKLM\System\CurrentControl
If above is true then to fix the issue you need to demote/promote the DC.You cannot demote the faulty DC gracefully you need to do forcefull removal.You need to ran dcpromo/force removal and then run matadata cleanup on other DC(healthy) to remove the instance of faulty DC from AD database and DNS.
Once done you can promote the Server back as DC.If faulty DC is FSMO role holder you need to seize the FSMO on other DC.
Reference link
Forcefull removal of DC: http://support.microsoft.com/kb/332199
Metadata cleanup: http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Seize FSMO role: http://www.petri.co.il/seizing_fsmo_roles.htm
Hope this helps
ASKER
Mike, that was the situation but I could not join DC1 back on to the domain. Should I have been using dcpromo instead of just trying to switch it from workgroup to domain? Situation now is that I reverted to snapshots of DC1 and DC2 so DC1 is still attached and DC2 is aware of DC1.
DC2 is servicing logon requests. Is FSMO role holder
DC1 has the DNS
DC1 is stuck in USN rollback
Question is, is it possible to have DC1 seize FSMO roles in this state and then demote DC2, or will I be left with no way to logon?
Thanks.
DC2 is servicing logon requests. Is FSMO role holder
DC1 has the DNS
DC1 is stuck in USN rollback
Question is, is it possible to have DC1 seize FSMO roles in this state and then demote DC2, or will I be left with no way to logon?
Thanks.
Why cant you create a new VM instead of using snapshots. DC2 not having DNS role in it ? I believe it should have DNS in it otherwise it cannot get logon requests.
First check DC2 is healthy and not having any issues by dcdiag /v. Shutdown the DC1.
Try to create a new VM with windows 2008 and promote it as DC. If you are able to promote it without any issues you don't have any issues with DC2. Don't mess things by using snapshots.
Move DHCP role by taking backup through netsh.
First check DC2 is healthy and not having any issues by dcdiag /v. Shutdown the DC1.
Try to create a new VM with windows 2008 and promote it as DC. If you are able to promote it without any issues you don't have any issues with DC2. Don't mess things by using snapshots.
Move DHCP role by taking backup through netsh.
ASKER
DC2 is servicing login. I know this because if it is off I cannot login to machines. DNS is on DC1 at moment though no reason not to move it and DHCP to DC2 anyway.
Best thing is to demote and repromote DC1 but I was spooked when it would not join back to the domain, even after metadata cleanup. What could have been preventing this?
Best thing is to demote and repromote DC1 but I was spooked when it would not join back to the domain, even after metadata cleanup. What could have been preventing this?
What error message you are recieving while adding the DC1 to domain?
ASKER
Sorry, did not note error message. Had not seen it before though and it was non-informative
If I were in your place I would have created new VM and promoted the server DC1 and ADC or if you still want the current VM server (DC1) to be DC.You first need to demote it forcefully and then promote the server back as DC assuming metadata cleanup of DC1 is done on DC2.
I will also recommend to run dcdiag /q and verify the health of DC before you proceed.
I will also recommend to run dcdiag /q and verify the health of DC before you proceed.
ASKER
Don't require DC1 to be a domain controller, just need to retain it on the domain as it has files and SQL.How do I demote DC1 but keep it on the domain as a domain server?
DC2 is FSMO role holder and now is the single DNS server and DHCP.
Presently creating new VM which will be DC3 (W2008). Finally, DC2 and DC3 will be domain controllers and DNS servers
DC2 is FSMO role holder and now is the single DNS server and DHCP.
Presently creating new VM which will be DC3 (W2008). Finally, DC2 and DC3 will be domain controllers and DNS servers
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
ASKER
I tried forcefully demoting but could not re-join the domain afterwards. I'll try again tomorrow after making sure that everything is running ok with the single DNS server on DC2.
If re-joining domain fails I'll investigate more fully. Possibly do this over the weekend so less pressure.
DC3 is not operational yet. I'll wait until DC1 is demoted otherwise I might have to clean metadata twice (DC2 and DC3)?
Suggestions appreciated. Thanks.
If re-joining domain fails I'll investigate more fully. Possibly do this over the weekend so less pressure.
DC3 is not operational yet. I'll wait until DC1 is demoted otherwise I might have to clean metadata twice (DC2 and DC3)?
Suggestions appreciated. Thanks.
If after forcefull demotion ensure that preferred dns setting on server is pointing to DC2 DNS server.If still issue persist post the error for further analysis.
As suggested verify the health of DC2 too.
As suggested verify the health of DC2 too.
Sounds like DC2 is up but with all FSMOs and DC1 is demoted/cleaned but you can't join it back. Is that right?
Thanks
Mike