jongrew
asked on
Active Directory Replication Problem
I have an problem whereby I have added new AD users and the Exchange email address is not being automatically added into the user’s details and so the users cannot access their mailboxes. This has started to happen on all new users we add.
I investigated the DCs and ran a NETDIAG and this is the output from it - it seems that Active Directory is not replicating properly and I was wondering if this would be the underlying cause. Is there any way to fix this issue?
Network topology as follows...
Head office = nt101 and nt102 and Exchange member server
Branch office 1 = nt811
Branch office 2 - nt1801
NETDIAG log file...
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.
C:\Documents and Settings\Administrator.MYD
Default-First-Site-Name\NT
DC Options: IS_GC DISABLE_INBOUND_REPL
Site Options: (none)
DC object GUID: 355d8465-5eb2-4432-b62c-a0
DC invocationID: 4728653e-6631-4fa2-bea8-35
==== INBOUND NEIGHBORS ==========================
DC=MYDOMAIN,DC=co,DC=uk
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 14:05:00 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
3394 consecutive failure(s).
Last success @ 2010-03-30 17:18:40.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 14:07:04 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
2704 consecutive failure(s).
Last success @ 2010-03-30 17:16:41.
CN=Configuration,DC=MYDOMA
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
833 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
820 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
CN=Schema,CN=Configuration
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
DC=DomainDnsZones,DC=MYDOM
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: 458b30be-a654-472c-9628-8c
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
319 consecutive failure(s).
Last success @ (never).
DC=ForestDnsZones,DC=MYDOM
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Source: Default-First-Site-Name\NT
******* 2694 CONSECUTIVE FAILURES since 2010-03-30 17:16:41
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
Source: Default-First-Site-Name\NT
******* 1 CONSECUTIVE FAILURES since 2010-04-22 13:53:07
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
Naming Context: CN=Schema,CN=Configuration
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: CN=Configuration,DC=MYDOMA
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=MYDOMAIN,DC=co,DC=uk
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=ForestDnsZones,DC=MYDOM
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Source: Default-First-Site-Name\NT
******* 3391 CONSECUTIVE FAILURES since 2010-03-30 17:18:40
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
I investigated the DCs and ran a NETDIAG and this is the output from it - it seems that Active Directory is not replicating properly and I was wondering if this would be the underlying cause. Is there any way to fix this issue?
Network topology as follows...
Head office = nt101 and nt102 and Exchange member server
Branch office 1 = nt811
Branch office 2 - nt1801
NETDIAG log file...
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.
C:\Documents and Settings\Administrator.MYD
Default-First-Site-Name\NT
DC Options: IS_GC DISABLE_INBOUND_REPL
Site Options: (none)
DC object GUID: 355d8465-5eb2-4432-b62c-a0
DC invocationID: 4728653e-6631-4fa2-bea8-35
==== INBOUND NEIGHBORS ==========================
DC=MYDOMAIN,DC=co,DC=uk
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 14:05:00 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
3394 consecutive failure(s).
Last success @ 2010-03-30 17:18:40.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 14:07:04 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
2704 consecutive failure(s).
Last success @ 2010-03-30 17:16:41.
CN=Configuration,DC=MYDOMA
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
833 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
820 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
CN=Schema,CN=Configuration
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:12.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
DC=DomainDnsZones,DC=MYDOM
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: 458b30be-a654-472c-9628-8c
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
319 consecutive failure(s).
Last success @ (never).
DC=ForestDnsZones,DC=MYDOM
Default-First-Site-Name\NT
DC object GUID: 830550bb-2dff-488e-9ad0-44
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
326 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Default-First-Site-Name\NT
DC object GUID: b1da112d-202a-404b-8d17-27
Last attempt @ 2010-04-22 13:46:17 failed, result 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
329 consecutive failure(s).
Last success @ 2010-03-30 16:58:13.
Source: Default-First-Site-Name\NT
******* 2694 CONSECUTIVE FAILURES since 2010-03-30 17:16:41
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
Source: Default-First-Site-Name\NT
******* 1 CONSECUTIVE FAILURES since 2010-04-22 13:53:07
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
Naming Context: CN=Schema,CN=Configuration
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: CN=Configuration,DC=MYDOMA
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=MYDOMAIN,DC=co,DC=uk
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=ForestDnsZones,DC=MYDOM
Source: Default-First-Site-Name\NT
******* WARNING: KCC could not add this REPLICA LINK due to error.
Source: Default-First-Site-Name\NT
******* 3391 CONSECUTIVE FAILURES since 2010-03-30 17:18:40
Last error: 8457 (0x2109):
Can't retrieve message string 8457 (0x2109), error 1815.
ASKER
Hi
I’m doing it as I always have done it which is to go onto the Exchange server directly as administrator using VNC and bring up AD Users and Computers snap-in and then adding a new user with an exchange mailbox when then after a short while e added the mailbox and email address into the users details. This has always worked up to a couple of days ago when the new user told me they could not access their mailbox.
I’m doing it as I always have done it which is to go onto the Exchange server directly as administrator using VNC and bring up AD Users and Computers snap-in and then adding a new user with an exchange mailbox when then after a short while e added the mailbox and email address into the users details. This has always worked up to a couple of days ago when the new user told me they could not access their mailbox.
Normally when the site link fails over time the only way to restart the replication is a physical restart of both servers.
You will normally in the event log a line saying that all problems preventing AD from replicating has been resolved and replication will commence.
I am unaware if you have the opportunity to restart this but also stoping and starting the netlogon service was recommended to me but never worked personally for me.
Im only saying this to rule out the restarts clearing it.
After which to test it open up the \\domaincontroller\netlogo
i use this as a basic replication test
You will normally in the event log a line saying that all problems preventing AD from replicating has been resolved and replication will commence.
I am unaware if you have the opportunity to restart this but also stoping and starting the netlogon service was recommended to me but never worked personally for me.
Im only saying this to rule out the restarts clearing it.
After which to test it open up the \\domaincontroller\netlogo
i use this as a basic replication test
ASKER
Without rebooting any servers I did the test as you described and added a test.txt file into the main DC and it was replicated without any input from me into all other DCs accross all sites within seconds. I did this several times with different text files from each DC and all replicated instantly.
ASKER
Further developments...
I have created a new RUS on the Exchange server and this one automatically avoided the bad DC and picked up one of the others on the network. I updated RUS and the SMTP addresses were added to the new AD users.
Another thing I have just noticed is that the new users I created are not being reflected in the first DCs AD users but is in the second and other DCs users. This confirms that the first DC is refusing updates for some reason. I also notice lack of disk space on the C drive of the first DC which could be causing an issue I suppose?
I have created a new RUS on the Exchange server and this one automatically avoided the bad DC and picked up one of the others on the network. I updated RUS and the SMTP addresses were added to the new AD users.
Another thing I have just noticed is that the new users I created are not being reflected in the first DCs AD users but is in the second and other DCs users. This confirms that the first DC is refusing updates for some reason. I also notice lack of disk space on the C drive of the first DC which could be causing an issue I suppose?
Hello,
Please open DNS and check which DC is 355d8465-5eb2-4432-b62c-a0
Also check this registry key: HKEY_LOCAL_MACHINE\SYSTEM\
Also, please let me know the GUIDs of all the 4 DCs.
Regards,
Arun.
Please open DNS and check which DC is 355d8465-5eb2-4432-b62c-a0
Also check this registry key: HKEY_LOCAL_MACHINE\SYSTEM\
Also, please let me know the GUIDs of all the 4 DCs.
Regards,
Arun.
from the NETDIAG it show that inbound replication has been disabled (DISABLE_INBOUND_REPL in DC Options) not sure what disabled it in first place ??
run
REPADMIN /options -DISABLE_INBOUND_REPL
on the DC experiencing the problem, it should then start replicating again if there is no problems.
if replication problems persist, the quickest solution would be to DCPROMO the machine out of teh domain, wait an hour or so for replication to get to all other DC's and then DCPromo it back in again.
run
REPADMIN /options -DISABLE_INBOUND_REPL
on the DC experiencing the problem, it should then start replicating again if there is no problems.
if replication problems persist, the quickest solution would be to DCPROMO the machine out of teh domain, wait an hour or so for replication to get to all other DC's and then DCPromo it back in again.
ASKER
I did run this command a couple of times last week to re-enable but it still shows disabled after. The DC experiecing the problem is the first DC and also a file server whith shares. Can i assume that when you run DC promo it will just go back to being a memner server or will it be in a workgroup. will the shares remain in place with all permissions etc?
ASKER
ARK-DS:
Check if "Netlogon" service is in Paused state??? - It is not paused
“Dsa Not Writable” does it have a value of 4? - It does have a value of 4
Server GUIDs
NT101 First DC - 355d8465-5eb2-4432-b62c-a0
NT102 - 458b30be-a654-472c-9628-8c
NT1801 - b1da112d-202a-404b-8d17-27
NT811 - 830550bb-2dff-488e-9ad0-44
Hope this helps
Check if "Netlogon" service is in Paused state??? - It is not paused
“Dsa Not Writable” does it have a value of 4? - It does have a value of 4
Server GUIDs
NT101 First DC - 355d8465-5eb2-4432-b62c-a0
NT102 - 458b30be-a654-472c-9628-8c
NT1801 - b1da112d-202a-404b-8d17-27
NT811 - 830550bb-2dff-488e-9ad0-44
Hope this helps
then the inbound replication was probably stopped by windows itslef due to some internal replication problem.
it will be domain member with permissions retained. during dcpromo it will ask you to reset the local administrator account for the server, take care to remember this so you can log in locally to the server after teh reboot if there is network problems that prevent it from finding a domain controller
1. Make sure that the DNS of the server you want to demote points to one of teh working domain controllers for DNS (i.e. does not point to itself)
2. when you DC promo down, it will become a member server of the domain ( do not unjoin it from the domain), and all domain security on files/folders will stay in place. Make sure that t
3. wait about 30 min for replication of other DC's to be done ( or force it with REPADMIN/REPLMON)
4. DCPromo server back in as domain controller
NOTE: make sure that when you DCPROMO it back in to the domain you use the default DOMAIN\Administrator user( not another user that has domain admins right) as the default user has specific rights that will resolve conflicts with the DCPROMO that other created domain admins will not have.
it will be domain member with permissions retained. during dcpromo it will ask you to reset the local administrator account for the server, take care to remember this so you can log in locally to the server after teh reboot if there is network problems that prevent it from finding a domain controller
1. Make sure that the DNS of the server you want to demote points to one of teh working domain controllers for DNS (i.e. does not point to itself)
2. when you DC promo down, it will become a member server of the domain ( do not unjoin it from the domain), and all domain security on files/folders will stay in place. Make sure that t
3. wait about 30 min for replication of other DC's to be done ( or force it with REPADMIN/REPLMON)
4. DCPromo server back in as domain controller
NOTE: make sure that when you DCPROMO it back in to the domain you use the default DOMAIN\Administrator user( not another user that has domain admins right) as the default user has specific rights that will resolve conflicts with the DCPROMO that other created domain admins will not have.
I had a gut feeling of this value being 4.
Can you tell me if this DC was restored ? I am asking this becasue most commonly DSA not writable becomes 4 if the DC is restored by using unsupported restoration procedures.
Anyways, although recommended way to deal with this situation is demotion and promotion but deleting DsaNotWritable key would also resolve the issue.
Note: If the DC was restored long time back OR the backup taken to restore the DC was quiete old then there are chance of you getting inconsistancies in the AD database. Thats why it is recommended to demote and promote the DC. If the backup was recent then I dont think there should be any problems deleting this value.
Regards,
Arun.
Can you tell me if this DC was restored ? I am asking this becasue most commonly DSA not writable becomes 4 if the DC is restored by using unsupported restoration procedures.
Anyways, although recommended way to deal with this situation is demotion and promotion but deleting DsaNotWritable key would also resolve the issue.
Note: If the DC was restored long time back OR the backup taken to restore the DC was quiete old then there are chance of you getting inconsistancies in the AD database. Thats why it is recommended to demote and promote the DC. If the backup was recent then I dont think there should be any problems deleting this value.
Regards,
Arun.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Your not adding the new users without the exchange administrator tool loaded on the server your adding the users too. as without this it will not display the exchange attributes or add the mailbox.