How to sync DC machine password after reset it?

Posted on 2010-01-07
Last Modified: 2012-05-08
We have an issue here, let me explian all happen
We have 18 branches, 21 domain controllers, the primary DC system  board crashed,  HP dispatched a tech to replace the board, when the server come back online fine, other DCs start to receive Kerberos error ID 4, which indecated the PDC machine password userd to encrpypt the Kerberos service ticket is different than that on the target server. The result of this is branch DC is not sync with this PDC, the replication topology is set up all branch DC sync with this PDC and vice vise.  And now helpdesk start to receiving calls about user changed password at branch , but not able to access portal and Citrix farm at datacenter, unless they use old password.

What I have been done
1) use netdom resetpwd to reset this PDC password against another DC See MS KB
2) these two DC can talk each other , but the reset password fail to sync to all other DCs, since other DC only talks to the PDC which is the one changed machine password.
3) try to create a new replication set between the one can talk to PDC with other branch DCs,  but it not replicate ( error " the naming context is in the process of being removed or is not replicated from the specified server)
4) ReplMon shows all replication are failing , except these two DC ( the PDC and reset password on other DC)

I am stuck here right now.
I have
 good backup of PDC before it went down.

Any advice will be very appreciated.
Question by:prcdiscover
    LVL 76

    Expert Comment

    Did you follow the directions included in the document you found and referenced with your question?
    I.e. did you stop the kerberos on all DC's experiencing issues step 2 Note? Then step 3 deleted cached kerberos credentials?
    Once you restore replication, you might want to consider the AD DC replication scheme from hub and spoke to all DCs can replicate with one another or if you have a specific VPN setup between location to setup replication between those that are already connected by a VPN.
    LVL 4

    Expert Comment

    Kerberos event 4 is not likely caused by the password of the PDC. Instead, there are two reasons that this may be logged:
    a duplicated SPN that is logged in the event;
    a duplicated IP address(A record) in DNS. i.e. in DNS two computers are having same IP address regisgtered.

    So, use the command below to check duplicated SPN:
    ldifde -f spn.txt -s ServerName -d "dc=domain,dc=com" -r "(objectclass=computer)" -l serviceprincipalname

    Also check DNS to see if another machine is having the same IP with the PDC.

    When user changes the password, it is synced firstly with the PDC. When the PDC is not reachable, the change failed and that is why the user can still use the old password. Again, this can be caused by duplicated IP address in DNS.

    If the issue persists and you have a good system state backup, try to backup the PDC and check again.

    Hope this helps!

    Author Comment

    Arnold and Hshao
    Thank you,

    Arnold, with MS support on line last night, we have made four out of 18 branches DC replicate now with the primary DC, we didnt try to stop all KDC services, ( we should),
    What we did
    While accessing comgt1 shares, we got target account name is incorrect error
    Downloaded and installed kerbtray
    Tested kerbtray and it failed to reset the secure channel
    Downloaded and installed klist
    We stopped KDC service, and disabled it
    Ran klist purge to purge the user tickets
    Ran the following 2 commands :
    netdom resetpwd /server:DC2 /userd:domain\administrator /passwordd:*

    net use \\DC2.domain.local\IPC$
    Forced the DC to replicate from DC2 by running repadmin /syncall /d /e
    Restarted KDC and set it to automatic
    Able to access \\PDC
    Verified replication monitor

    the questions come from me
    1) Does one server( in this case a DC) only have one machine password through entire forest? I believe the answer is true,
    2) if 1) is yes, then everytime I ran netdom from the PDC against a DC, the machine password for the PDC will changed, and try to sync to all other DC
    3) according to the MS KB, KDC need to stop on all DCs, will this affect user login, anyother impacts?
    Funny is the MS support doesnt know this KB, I am just showing her ( from India) the KB about all KDC service on all DCs have to stop.
    Will keep this updated,

    Author Comment

    Forgot to add one more thing,
    This morning , all branch user can not access exchange through Outlook, they still can access through webmail.
    Portal and other shared folder from HQ site are still accessible.
    Dont understand why cant exchange through outlook at the mean time webmail is fine, this happen on all branches, include the four branchs' DC can replicate with PDC
    LVL 76

    Expert Comment

    What is the error that the outlook users get?
    If you manage outlook settings via GPO, use GPMC to get a Policy Result set to see what is happening i.e. a GPO policy dealing with outlook settings is not applying.

    Author Comment

    All the users except from HQ are prompted for the user name and password, when the user type the correct password, it still prompted, and it prompted to different server ( switch between the exchange backend server and primary DC) , the webmail, which is point to the exchange front server is working fine. ( I am worry about this may stop working anytime)
    It looks like the exchange backend server kerberbos machine ticket timeout,
    Not sure,
    LVL 76

    Expert Comment

    Unfortunately, in your situation there could be two possible "correct passwords."
    The new and the old.  Try both. Does the user's account get locked.
    Which DC  does the exchange Front Server is use to authenticate users?

    The issue could have been the result of the replication model you have just for a case where the HUB goes down, each child has their own recent changes they want to push.
    The other possibility is that you have a machine password change requirement that kicked in within the same time frame the server malfunctioned. All speculation what led to this.  I think the kerberos is only used to setup the secure channel from DC to DC to exchange data.
    At this point you have four or five out of the 21 that resumed replication.
    Was the suggestion to look for a duplicate IP helpful in solving this issue?
    You might as well go one at a time to the DCs that do not sync and clear their machine ticket and reboot based on the article you found or using the process you did with MS support if it is different.


    Author Comment

    I have talked to two users, they havent change password for more than a month.
    both frontend and backend server are authenticated by the secondary DC on HQ site, when I type set l, it shows the secondary DC, but how about user, when user authenticated by exchange, how the aithentication topology works, I did mention, when user keep type the user name and password, it actaully prompted different servers, one to backend server, then PDC, then backend server again, then PDC.  so not sure how to change the authentication server when user login through Outlook.

    There is no duplicate IP address on the network. I have double checked.

    I may want to do stop KDC on all DCs at the same time, then reset it, but I not sure the possible impact,
    LVL 76

    Accepted Solution

    The only caution deals with not stopping KDC on GC server/s
    Do you have branches with multiple DC?
    The other issue is that the branches are already in a strange state. Try stopping the KDC on two currently not replicating DC and perform the instructions included in the article you posted and see whether that brings both in without adversely affecting the four that already sync at the same time.  If it does, you can try the same with five or six. And then the remainder. Double check your replication topology setup once AD replication is restored.

    Author Comment

    This issue has been fixed.
    Basically, the KB article need to follow

    1)      stop kdc on all DCs have issue
    2)      start kerbtray, purge ticket one by one,
    3)      reset password  netdom resetpwd /server:localserver /userd: yourdoamin\\administrator /passwordd:*
    4)      start \\pdc wait couple of minutes
    5)      start kdc  on all DCs wait couple of minutes
    6)       run  repadmin /syncall /d /e from PDC

    Some of the site still failed with same error “ The target principal name is incorrect”

    Keep  re run this on the failed DC, until it fixed.

    Thank you all for the help.


    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    by Nathan Brom/Bromy2004 Introduction There are numerous websites out there for any different type of program you can imagine.  Of those, you'll need to decide which ones are legitimate and aren't trying to steal your money or infect your comput…
    Many times while working on a computer regardless of any Operating System, lag and crashes seem to creep in, hindering your working speed. Sometimes, it can also cause your work to be lost unexpectedly and as a result, you are unable to meet your de…
    Windows 8 comes with a dramatically different user interface known as Metro. Notably missing from the new interface is a Start button and Start Menu. Many users do not like it, much preferring the interface of earlier versions — Windows 7, Windows X…
    In this video, we discuss why the need for additional vertical screen space has become more important in recent years, namely, due to the transition in the marketplace of 4x3 computer screens to 16x9 and 16x10 screens (so-called widescreen format). …

    732 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    23 Experts available now in Live!

    Get 1:1 Help Now