Active Directory removing default permissions on user objects

First of all how many domain controllers you have in your network? Present the structure of your folder you are trying to apply permissions to.

Greetings,

There are currently 11 domain controllers in our network.  Two at five remote locations and a single one at a sixth.  All of these locations are defined as sites.  

The object I am trying to change the permissions is under the following structure...

ourdomain.com
     -locationOU (ie Illinois)
               -Users
                         -UserObject

As for the up to dateness vector...I think you may be on to something.  I have never dug that deep into AD replication...so unfortunately I am a little lost.  Here is the output of that command.

Niles\DRACO                          @ USN 6227149
FortWorth\EVEE                       @ USN 138463
3830e535-c128-4dd5-9a17-e64ec55c9a18 @ USN 1493514
ElkGrove\ARIEL                       @ USN 4364262
60cb7635-c089-42a8-b1f9-28edc5d95894 @ USN 383148
Fountaininn\FIFS                     @ USN 2061252
Niles\VELA                           @ USN 7895258
6764d884-115e-4eb6-95ef-c76db23f85b4 @ USN 3853366
Brunswick\BRFS                       @ USN 1232714
ElkGrove\CITRIXUTIL                  @ USN 9466863
Fountaininn\FI_ADC1                  @ USN 3152757
af8a18c8-9e6d-438f-8389-9a98abdde555 @ USN 684261
b209aebb-6481-4c0c-876a-0f9cb3639277 @ USN 817834
FortWorth\FWFS                       @ USN 155965
cb99c0c6-1886-4638-bca6-66549f7bfdb0 @ USN 2602641
Brunswick\BRUNSWICK                  @ USN 3109191
d680cdee-85fa-443c-bda7-da010dcac4f3 @ USN 466776
Philly\FILES                         @ USN 43872

I think we are on the right track.  A little more guidance and I think I'll be able to track down the problem and fix it.

Thanks for all the help so far!!!

One more thing to add...this morning I watched my entire AD.  I changed the object at about 8:09am and watched it replicate out to all my domain controllers.  It appeared to take.  About 1 hour later...something rolled it back.  I replicate every 15 minutes...which means all of my domain controllers should have had the changes within 30 minutes.  I do not understand this at all.

Ok ... as a simple and quick response ...

do you see the USN  numbers? well ... what you have to accomplish is the following.

Say that you apply the settings to the user objects on the DC BRUNSWICK ... you have to manually up the number to make it greater than all the other USN, so its settings take presedence over every other DC in the whole forest.

You got to be carefull though. usually AD does this automaticlly avoiding certain errors. Some errors by doing this manually could be the fact that you might lose any changes made in the FWFS location, becuase the BRUNSWICK location with the highest USN number, overwrites anything on qany other server ... AD usually automaticly identifies if there is a conflict between the two versions and tries to make an arrangment between both replications as to not lose both configurations ...

What i mean, is that say that you change password for user1 to "X" on DC1 - and then you change the password for user1 on DC2 to "xx", which one would take presedence over the other?

AD automaticlly resolves the issue.

Now, if you manually up the USN of the DC, no matter what, the manual one takes presedence over the other. Unless its some other type of setting, like creating a user on one DC and changing password on the other. Even if one of them has higher USN, becuase they don't conflict, both changes are made effective ...

Now, I know many expert exchange pros are gonna come up and kill me for stating this becuase it doesn't really occur this way. But I am putitng it that way for you to understand what I am trying to acomplish here ...

for more info, on what takes effectivnes over the other, search and read about replication. Any site specially at microsoft covers this subject throughly ... for example:

Anyways, you basiclly got to dig into that, and up that number alot of times, enough to pass any other USN, so when replication occurrs, and you got to make sure it gets to every DC, you'll make sure that the changes finally apply ...

You should be able to start the replication just by upping that number ...

or maybe something like this: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=22007 or http://www.windowsitpro.com/Article/ArticleID/43984/43984.html

Just in case you could monitor replication using "repadmin"
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/TechRef/03b7fc47-e25c-4af8-822f-f856b565b76a.mspx

cheers!

Hope you understand where I am going ...

"One more thing to add...this morning I watched my entire AD.  I changed the object at about 8:09am and watched it replicate out to all my domain controllers.  It appeared to take.  About 1 hour later...something rolled it back.  I replicate every 15 minutes...which means all of my domain controllers should have had the changes within 30 minutes.  I do not understand this at all."

This really depends on replication topology ... it oculd take more time, and replication could occur at different times on different DC ... making these type of errors to posibly occur ...

Try forcing it and lets see what happens ...

OK,

So far I do understand where this is going.  I have three more questions though.  Sorry...I am trying to get this as quickly as possible.

1.  How do I manually change the USN for a domain controller?

2.  What are the servers that don't appear to have names/sites...just IDs?  And can I clean them out???

3.  Elk Grove/Citrixutil appears to have the largest USN...however when I make the changes on it...they still get rolled back.  

I think I am just one more post short of finally getting it.  The lightbulb just hasn't turned on yet...i am still missing something.

Thanks for all the answers so far!!!

OK...I have more info.  Using repadmin...I watched one of the objects that keeps resetting itself.  I made the changes to the object on a server named ariel.  Using repadmin /showmeta....i got the info for the object I changed.  Here is output...please note the NTSecurityDescriptor field...

36 entries.

Loc.USN                      Originating DSA Org.USN       Org.Time/Date  Ver Attribute
=======                      =============== =======       =============  === =========
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectClass
   9706                       ElkGrove\ARIEL    9706 2003-09-09 14:47.08    1 cn
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sn
 406343                           Niles\VELA 4857462 2004-01-05 14:25.35    3 description
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 givenName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 instanceType
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 whenCreated
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 displayName
4364944                       ElkGrove\ARIEL 4364944 2005-06-28 10:29.17   18 nTSecurityDescriptor
3348969                       ElkGrove\ARIEL 3348969 2005-02-10 10:12.34    5 name
2178878                       ElkGrove\ARIEL 2178878 2004-08-16 11:00.37    4 userAccountControl
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 codePage
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 countryCode
1172621                       ElkGrove\ARIEL 1172621 2004-05-03 09:20.10    2 homeDirectory
1172621                       ElkGrove\ARIEL 1172621 2004-05-03 09:20.10    2 homeDrive
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 dBCSPwd
2016532                  ElkGrove\CITRIXUTIL 5042306 2004-07-21 13:46.31    2 scriptPath
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 logonHours
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 userWorkstations
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 unicodePwd
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 ntPwdHistory
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 pwdLastSet
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 primaryGroupID
4050409                  ElkGrove\CITRIXUTIL 9226222 2005-05-31 11:14.43    6 supplementalCredentials
4348476                  ElkGrove\CITRIXUTIL 9413753 2005-06-24 10:45.48    4 userParameters
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 profilePath
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectSid
3514147                       ElkGrove\ARIEL 3514147 2005-03-08 12:58.07    1 adminCount
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 comment
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 accountExpires
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 lmPwdHistory
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sAMAccountName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sAMAccountType
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 userPrincipalName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectCategory
   9706                           Niles\VELA 3176178 2003-06-13 09:51.44    1 msNPAllowDialin


After a short while...i checked and sure enough...the settings had reset themsevles.  I ran the same command on the same server (ariel) to get the properties and ....

36 entries.

Loc.USN                      Originating DSA Org.USN       Org.Time/Date  Ver Attribute
=======                      =============== =======       =============  === =========
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectClass
   9706                       ElkGrove\ARIEL    9706 2003-09-09 14:47.08    1 cn
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sn
 406343                           Niles\VELA 4857462 2004-01-05 14:25.35    3 description
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 givenName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 instanceType
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 whenCreated
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 displayName
4365070                       ElkGrove\ARIEL 4365070 2005-06-28 10:59.26   19 nTSecurityDescriptor
3348969                       ElkGrove\ARIEL 3348969 2005-02-10 10:12.34    5 name
2178878                       ElkGrove\ARIEL 2178878 2004-08-16 11:00.37    4 userAccountControl
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 codePage
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 countryCode
1172621                       ElkGrove\ARIEL 1172621 2004-05-03 09:20.10    2 homeDirectory
1172621                       ElkGrove\ARIEL 1172621 2004-05-03 09:20.10    2 homeDrive
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 dBCSPwd
2016532                  ElkGrove\CITRIXUTIL 5042306 2004-07-21 13:46.31    2 scriptPath
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 logonHours
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 userWorkstations
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 unicodePwd
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 ntPwdHistory
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 pwdLastSet
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 primaryGroupID
4050409                  ElkGrove\CITRIXUTIL 9226222 2005-05-31 11:14.43    6 supplementalCredentials
4348476                  ElkGrove\CITRIXUTIL 9413753 2005-06-24 10:45.48    4 userParameters
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 profilePath
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectSid
3514147                       ElkGrove\ARIEL 3514147 2005-03-08 12:58.07    1 adminCount
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 comment
   9706                           Niles\VELA 3176170 2003-06-13 09:51.35    1 accountExpires
4050411                       ElkGrove\ARIEL 4050411 2005-05-31 11:14.42    8 lmPwdHistory
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sAMAccountName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 sAMAccountType
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 userPrincipalName
   9706                           Niles\VELA 3176169 2003-06-13 09:51.35    1 objectCategory
   9706                           Niles\VELA 3176178 2003-06-13 09:51.44    1 msNPAllowDialin

you will notice the NTSecurityDescriptor has been updated....by Ariel itself.  It appears that ariel is rolling back the changes I make to certain objects...even if I make the changes on ariel in the first place.

Two things to note...Ariel is running win2k/sp4 (as are all of my DCs) and it is also the operations master for the domain.

Any further ideas?????

Thank you in advance if you have the solution to this problem.

Hmm , I Would Suggest to check where the fsmo roles are.
could be you have more than one fsmo role in the domain?
maybe someone seized them?

Actually,

All five fsmo roles are assigned to the server that keeps resetting the properties (ariel).  They were moved from an older server that has since been decommisioned.  As far as I know...they have never been assigned to or seized from any other server.

could you possibly fit into this picture?
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/Operations/4a1f420d-25d6-417c-9d8b-6e22f472ef3c.mspx

Is there any errors in the Directory Service event log?


Please not that although ARIEL has a high USN these server have higher USN ...

ElkGrove\CITRIXUTIL
Niles\VELA
Niles\DRACO

Try originating the change from one of those server, and see what happens ...

Also, run a repadmin failcache

It Displays a list of failed replication events that are detected by Knowledge Consistency Checker (KCC).

Syntax
repadmin /failcache

Can you also copy over the results of a "repadmin /showrepl"?

OK...

I do not belive we have lingering objects...though we are running 2000 here and that article doesn't really have a method for checking/cleaning a 2000 AD.  The objects that are having the issue appear to be random throughout the directory and have always been there.  I do not believe any of them have ever been deleted/recreated or anything like that.

I have made changes from the other servers with no luck.  I can see that the initial change was inititated by whichever server I made the change from...and it does actually replication out to all of my remote domain controllers...but then 30-60 minutes laters...the changes roll back and AD shows ariel made them.

I also ran a repadmin /failcache.  The only error was a connection error with a test child domain I setup a while ago.  That domain controller still exist, but is offline.  I can run a dcpromo if and remove it if anybody thinks that may help.

And finally...repadmin /showrepl does not exist on 2000.  That is a 2003 command.  The closest I could find was repadmin /showreps.  Here is the output for Ariel...

C:\Documents and Settings\sysadmin>repadmin /showreps dc=fortdearborn,dc=com ari
el.fortdearborn.com
ElkGrove\ARIEL
DSA Options : (none)
objectGuid  : 7314acd0-8040-4950-8788-29e9ea751a7d
invocationID: 4c55c98d-47a2-4a7c-902b-5f2a85a79015

==== INBOUND NEIGHBORS ======================================

dc=fortdearborn,dc=com
    ElkGrove\CITRIXUTIL via RPC
        objectGuid: f8f05160-653c-472b-bdaf-40bacd99d315
        Last attempt @ 2005-06-29 07:25.19 was successful.

==== OUTBOUND NEIGHBORS FOR CHANGE NOTIFICATIONS ============

dc=fortdearborn,dc=com
    ElkGrove\CITRIXUTIL via RPCC:\Documents and Settings\sysadmin>repadmin /showreps dc=fortdearborn,dc=com cit
rixutil.fortdearborn.com
ElkGrove\CITRIXUTIL
DSA Options : IS_GC
objectGuid  : f8f05160-653c-472b-bdaf-40bacd99d315
invocationID: 7708ad9e-b79a-463e-ae1c-b25804c38108

==== INBOUND NEIGHBORS ======================================

dc=fortdearborn,dc=com
    Brunswick\BRUNSWICK via RPC
        objectGuid: 019aaa56-c161-43b5-bcdd-384feab28e57
        Last attempt @ 2005-06-29 07:30.16 was successful.
    FortWorth\EVEE via RPC
        objectGuid: abffe929-a607-4015-b1fd-99d2ea3af132
        Last attempt @ 2005-06-29 07:30.16 was successful.
    Fountaininn\FIFS via RPC
        objectGuid: 4c816b13-1297-447f-9d18-5d28df79f3d4
        Last attempt @ 2005-06-29 07:30.17 was successful.
    Niles\VELA via RPC
        objectGuid: 66b4d9bd-acc5-4e0e-b2b1-8c2e69705059
        Last attempt @ 2005-06-29 07:30.17 was successful.
    Philly\FILES via RPC
        objectGuid: d0365799-3896-4a2e-843f-f5d16fd3f44e
        Last attempt @ 2005-06-29 07:30.17 was successful.
    ElkGrove\ARIEL via RPC
        objectGuid: 7314acd0-8040-4950-8788-29e9ea751a7d
        Last attempt @ 2005-06-29 07:33.13 was successful.

==== OUTBOUND NEIGHBORS FOR CHANGE NOTIFICATIONS ============

dc=fortdearborn,dc=com
    ElkGrove\ARIEL via RPC
        objectGuid: 7314acd0-8040-4950-8788-29e9ea751a7d
        objectGuid: f8f05160-653c-472b-bdaf-40bacd99d315

Here is the output then for citrixutil...

Please let me know if anyone has more info.

Thanks for all the help!!!

Are there any errors in the Directory Service event log ...

how to activate it: http://support.microsoft.com/default.aspx?scid=kb;en-us;314980&sd=tech

how to view it: http://support.microsoft.com/?kbid=235427 (in your case ignore the "remote viewing". I just posted this up for you to know where the log is.)

As for this: "I also ran a repadmin /failcache.  The only error was a connection error with a test child domain I setup a while ago.  That domain controller still exist, but is offline.  I can run a dcpromo if and remove it if anybody thinks that may help."

This could be the problem. What was the Test DC Computer name? Does it appear in the inbound or outbound list of any DC?

------------------

Check this out:

Display pending replication changes.

Each domain controller keeps a record of the last changes received from other domain controllers by using the Update Sequence Number (USN) of the replication partner. In this example changes have occurred on the replication partner and the Active Directory replication process has not propagated those changes to the monitored server. You can use ReplMon to show what objects have changed and therefore need to be replicated to the monitored server. This is done on a per-directory partition basis.

To show what objects have changed

1. In ReplMon, expand a server to display the list of directory partitions.
 
2. Expand and select the replication partner within the directory partition for which you want to view unreplicated changes. Only objects that have changed within the directory partition are displayed.
 
3. On the Action menu, click Replication Partner, and then select Check Current USN and Unreplicated Objects.

A separate window opens. Detected changes are displayed and can be saved to a text file. If no changes are detected, a message box informs you of the current USN on the replication partner and that all changes have replicated to the monitored server.
 

------------------------------------------
Try out the above to see whats up with the USN ... and see if any DC has unreplicated objects comparing it to Ariel server where you are making the changes ...
------------------------------------------

Try resyncing with correct AD settings using this method. Do it with all partitions and to all DC. I can't understand why your settings only apply for 30 minutes. Are you sure that you make the change on Ariel and that ariel is the one who reverts the change? Becuase my theory is that due to replication problems you make the change at ariel but becuase theres another conflict with another server the change is being discarted ... but by that other server .. not the same ...

Example 4: Synchronize directory partitions

In this example, you use ReplMon to force a replication event between two directory partitions. By default, directory partitions on servers periodically synchronize with each other. Manual synchronization is necessary only if the network or a server is down for an extended period of time.

To manually synchronize two directory partitions

1. In the Monitored Servers pane of ReplMon, select the server that needs to be brought up to date.
 
2. Expand a directory partition.

A list of replication partners for that directory partition appears.
 
3. Select the directory partition that needs to be synchronized.
 
4. On the Action menu, click Synchronize with this Replication Partner.
 
5. In the dialog box that appears, select any additional parameters for synchronization, then click OK to continue with the synchronization.
 
• This method synchronizes only one directory partition at a time. To synchronize all directory partitions at once, on the Action menu, click Server, and then click Synchronize Each Directory Partition with All Servers.


-------------*-----------------

Could you posibly have a 3rd party program that might be setting back the changes?

Thank you for all the advice.  I won't be able to get to all of this until after the holiday weekend...but i am not abandoning this question until i have a solution!!!

if I run a repadmin /showmeta (I think that is the command...it is something like that anyway) on the object I change...I see for the nTSecurityDescripter property the originating DSA changes to whatever server i made the change on.  Every time that change reverts back...the originating DSA changes to Ariel.  That is why i believe Ariel is responsible.

At any rate...I am going to remove the testdc from the network and see if that helps...though I kind of doubt it.

I will get to the monitoring unreplicated changes/resyncing next week.  Also...I do not believe I have seen any Directory Services errors in the log...but I will keep an eye out and report this for sure next week.

Thanks again for all the help.

but remember you can like activate details AD logging ...

Check out the link ... I will await the results ...

OK,

This gets even weirder.  I made changes to one of the objects I have been having trouble with and watched it propagate by running repadmin /showmeta "object......."  I made the initial changes on a server called Citrixutil.

I also turned on DS logging on ariel...the server i suspect is causing all of my problems.

here are excerpts of the repadmin /showmeta command run on ariel after I made the change to that object and after the changes were reverted back...

AFTER THE CHANGES WERE MADE...
4408883                  ElkGrove\CITRIXUTIL 9573438 2005-07-07 10:12.27   22 nTSecurityDescriptor

AFTER THEY REVERTED BACK...
4408993                       ElkGrove\ARIEL 4408993 2005-07-07 10:39.04   23 nTSecurityDescriptor

Here is the output of the ds log for 10:39:04

7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 8b082e20-fa9d-4f51-aed1-78100e7559c2 from USN 39468 to 39509 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 7708ad9e-b79a-463e-ae1c-b25804c38108 from USN 9573579 to 9573678 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 696612d6-f3fb-4451-aa9d-a7fbefcecfcd from USN 1255731 to 1255754 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 66b4d9bd-acc5-4e0e-b2b1-8c2e69705059 from USN 7935351 to 7935392 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 62d759d5-d88e-4bbe-98ee-c2950bb66cc9 from USN 2105666 to 2105715 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 2b76a2ab-ea7e-4887-bdd4-a7403f8319fe from USN 178735 to 178780 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 072b1175-2998-4c1f-a42f-55f463db4e5c from USN 794219 to 794233 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 008259f8-8ebb-4406-b31c-980c8584c0a8 from USN 6259325 to 6259370 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1366      Everyone      ARIEL      Internal event: Applying update of object CN=NTDS Site Settings,CN=Niles,CN=Sites,CN=Configuration,DC=fortdearborn,DC=com with objectGuid 0d5ce577-0717-495a-8017-a2072cefef44.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1412      Everyone      ARIEL      Property 904de (interSiteTopologyGenerator) of object CN=NTDS Site Settings,CN=Niles,CN=Sites,CN=Configuration,DC=fortdearborn,DC=com (GUID 0d5ce577-0717-495a-8017-a2072cefef44) is being applied to the local database. The remote version is 63304, timestamp 2005-07-07 10:33.32 and originating USN 7935387.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1413      Everyone      ARIEL      Property 20001 (instanceType) of object CN=NTDS Site Settings,CN=Niles,CN=Sites,CN=Configuration,DC=fortdearborn,DC=com (GUID 0d5ce577-0717-495a-8017-a2072cefef44) is not being applied to the local database because its local metadata implies the change is redundant.  The local version is 1.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Configuration,DC=fortdearborn,DC=com from directory ID f8f05160-653c-472b-bdaf-40bacd99d315 with options 55.
7/7/2005      10:39:04 AM      NTDS Replication      Warning      Replication       1061      Everyone      ARIEL      Internal error: The directory replication agent (DRA) call returned error 1753.
7/7/2005      10:39:04 AM      NTDS Replication      Warning      Replication       1085      Everyone      ARIEL      Replication warning: The directory replication agent (DRA) couldn't synchronize partition CN=Configuration,DC=fortdearborn,DC=com with partition on directory server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com.
 
 The error was:
 There are no more endpoints available from the endpoint mapper.
 
 Please verify that the address can be resolved with DNS, and that it is reachable via the transport.  If this error persists, the KCC will reconfigure the links around this server.
 The record data is the status code.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1124      Everyone      ARIEL      Internal event: The directory replication agent (DRA) failed to get a remote procedure call (RPC) binding handle for server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com, error 1753.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Configuration,DC=fortdearborn,DC=com from directory ID a509f7c2-3b12-45ac-a459-b3742c2ed30d with options 55.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1060      Everyone      ARIEL      Internal event: The directory replication agent (DRA) call completed successfully.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1364      Everyone      ARIEL      Internal event: Improving the USN vector for DSA f8f05160-653c-472b-bdaf-40bacd99d315 from (9573598/OU, 9573598/PU) to (9573673/OU, 9573673/PU) for partition DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 7708ad9e-b79a-463e-ae1c-b25804c38108 from USN 9573673 to 9573678 for partition DC=fortdearborn,DC=com.
7/7/2005      10:39:04 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of DC=fortdearborn,DC=com from directory ID f8f05160-653c-472b-bdaf-40bacd99d315 with options 55.
7/7/2005      10:39:04 AM      NTDS SDPROP      Information      Internal Processing       1260      N/A      ARIEL      The Security Descriptor Propagator is waiting for a propagation event.
7/7/2005      10:39:04 AM      NTDS SDPROP      Information      Internal Processing       1258      Everyone      ARIEL      The directory serviced processed security descriptor propagation number 23.  1 objects were touched.
7/7/2005      10:39:04 AM      NTDS SDPROP      Information      Internal Processing       1257      Everyone      ARIEL      The directory service processing security descriptor propagation number 23 starting from node 2611.
7/7/2005      10:39:04 AM      NTDS SDPROP      Information      Internal Processing       1261      N/A      ARIEL      The Security Descriptor Propagator has been notified of waiting propagation events.
7/7/2005      10:39:04 AM      NTDS General      Information      Directory Access       1174      Everyone      ARIEL      A privileged operation (rights required = 0x) was successfully performed on object S-1-5-21-606747145-920026266-1801674531-4695.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.
7/7/2005      10:39:04 AM      NTDS Database      Information      Internal Processing       1167      Everyone      ARIEL      The directory has elected INDEX_00090092 as the optimal index for this query.

If you would like the entire file...i would be happy to send them to you.

Thank you again very much for all your help.

Note...i am upping the points as this seems to be a very difficult issue.

ok. I need you to run a DCDIAG on the ARIEL Server .. and check out what errors it throws back at you...

You should also check this out:

1. Verify the status and startup type for the following services on the server that gets the error:
Type of computer                                            -   RPC service           - RPC Locator service
Windows Server 2003-based domain controller -  Started, Automatic   - Stopped, Manual
Windows Server 2003-based member server     - Started, Automatic   - Stopped, Manual
Windows Server 2003-based standalone server  -  Started, Automatic   -  Stopped, Manual
Windows 2000 Server-based domain controller   -  Started, Automatic   -  Started, Automatic
Windows 2000 Server-based member server     -  Started, Automatic   -   Started, Manual
Windows 2000 Server-based standalone server  -  Started, Automatic   -  Stopped, Manual

If you make any changes to the RPC service or to the RPC Locator service settings, restart the computer, and then test for the problem again...

this error worries me:
10:39:04 AM     NTDS Replication     Information     Replication      1413     Everyone     ARIEL     Property 20001 (instanceType) of object CN=NTDS Site Settings,CN=Niles,CN=Sites,CN=Configuration,DC=fortdearborn,DC=com (GUID 0d5ce577-0717-495a-8017-a2072cefef44) is not being applied to the local database because its local metadata implies the change is redundant.  The local version is 1.

hmmm, I'll wait for that DCDIAG and lets see if fixing the RPC on ariel gets replication back up and running correctly ...

Now we may be getting some where!!!

1.  I checked the RPC Service and RPC Locator Service...ariel is a windows 2000 DC and both were set to started, automatic as you posted.

2.  I ran the dcdiag test on ariel...please see the results...and errors...below.

I will wait for your your next instructions...thank you sooo much for all your help with this issue!!!

Domain Controller Diagnosis

Performing initial setup:
   Done gathering initial info.

Doing initial required tests

   Testing server: ElkGrove\ARIEL
      Starting test: Connectivity
         ......................... ARIEL passed test Connectivity

Doing primary tests

   Testing server: ElkGrove\ARIEL
      Starting test: Replications
         ......................... ARIEL passed test Replications
      Starting test: NCSecDesc
         ......................... ARIEL passed test NCSecDesc
      Starting test: NetLogons
         ......................... ARIEL passed test NetLogons
      Starting test: Advertising
         ......................... ARIEL passed test Advertising
      Starting test: KnowsOfRoleHolders
         ......................... ARIEL passed test KnowsOfRoleHolders
      Starting test: RidManager
         ......................... ARIEL passed test RidManager
      Starting test: MachineAccount
         ......................... ARIEL passed test MachineAccount
      Starting test: Services
         ......................... ARIEL passed test Services
      Starting test: ObjectsReplicated
         ......................... ARIEL passed test ObjectsReplicated
      Starting test: frssysvol
         ......................... ARIEL passed test frssysvol
      Starting test: kccevent
         An Information Event occured.  EventID: 0x40000497
            Time Generated: 07/08/2005   08:33:11
            (Event String could not be retrieved)
         ......................... ARIEL failed test kccevent
      Starting test: systemlog
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   07:43:49
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   07:43:49
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   07:55:02
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   07:55:02
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:08:05
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:08:05
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:32:45
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:32:45
            (Event String could not be retrieved)
         ......................... ARIEL failed test systemlog

   Running enterprise tests on : fortdearborn.com
      Starting test: Intersite
         ......................... fortdearborn.com passed test Intersite
      Starting test: FsmoCheck
         ......................... fortdearborn.com passed test FsmoCheck

I ran it again a few minutes later....more errors!!!


   Testing server: ElkGrove\ARIEL
      Starting test: Connectivity
         ......................... ARIEL passed test Connectivity

Doing primary tests

   Testing server: ElkGrove\ARIEL
      Starting test: Replications
         ......................... ARIEL passed test Replications
      Starting test: NCSecDesc
         ......................... ARIEL passed test NCSecDesc
      Starting test: NetLogons
         ......................... ARIEL passed test NetLogons
      Starting test: Advertising
         ......................... ARIEL passed test Advertising
      Starting test: KnowsOfRoleHolders
         ......................... ARIEL passed test KnowsOfRoleHolders
      Starting test: RidManager
         ......................... ARIEL passed test RidManager
      Starting test: MachineAccount
         ......................... ARIEL passed test MachineAccount
      Starting test: Services
         ......................... ARIEL passed test Services
      Starting test: ObjectsReplicated
         ......................... ARIEL passed test ObjectsReplicated
      Starting test: frssysvol
         ......................... ARIEL passed test frssysvol
      Starting test: kccevent
         An Warning Event occured.  EventID: 0x8000043D
            Time Generated: 07/08/2005   08:54:10
            (Event String could not be retrieved)
         An Warning Event occured.  EventID: 0x80000425
            Time Generated: 07/08/2005   08:54:10
            (Event String could not be retrieved)
         An Warning Event occured.  EventID: 0x8000043D
            Time Generated: 07/08/2005   08:54:11
            (Event String could not be retrieved)
         An Warning Event occured.  EventID: 0x80000425
            Time Generated: 07/08/2005   08:54:11
            (Event String could not be retrieved)
         An Information Event occured.  EventID: 0x40000497
            Time Generated: 07/08/2005   08:54:21
            (Event String could not be retrieved)
         An Information Event occured.  EventID: 0x40000497
            Time Generated: 07/08/2005   08:55:37
            (Event String could not be retrieved)
         An Warning Event occured.  EventID: 0x800004C0
            Time Generated: 07/08/2005   08:57:27
            (Event String could not be retrieved)
         ......................... ARIEL failed test kccevent
      Starting test: systemlog
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:53:37
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:53:37
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:56:38
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:56:38
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:57:02
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0x00003004
            Time Generated: 07/08/2005   08:57:02
            (Event String could not be retrieved)
         ......................... ARIEL failed test systemlog

   Running enterprise tests on : fortdearborn.com
      Starting test: Intersite
         ......................... fortdearborn.com passed test Intersite
      Starting test: FsmoCheck
         ......................... fortdearborn.com passed test FsmoCheck

The best I could find on your problem is this site:

http://www.microsoft.com/technet/prodtechnol/windowsserver2003/library/Operations/d861ac41-4ad3-4468-bace-7ef5b6421bf2.mspx

I cannot find anything specific to your problem ...

I will keep on looking. Take a look at the link, meanwhile I'll keep on looking and post up more material ...

BRB

Try disabiling ... http://support.microsoft.com/?kbid=242780 and re-enabiling ... "/enable" the KCC and lets see if that helps ...

I honestly cannot find any detials on the eventID you gave me ...

Very weird ... I'll keep on looking though .,..

Thanks for the tip...I'll give it a try over the weekend and get back to you on Monday!!!

OK .. great. I'll try to get more resources on this as soon as posible meanwhile.

Hey UICE,

thank you soo much for all your help.  I kicked the points up to 400 if we can get to the bottom of this.

Anyway...I wanted to provide some additional info/results.  I have been going over all the posts and try everything again.

Here is what I noticed.  Here is another snippet of the log from ariel after it reversed changes

7/11/2005      8:41:12 AM      NTDS Replication      Warning      Replication       1085      Everyone      ARIEL      Replication warning: The directory replication agent (DRA) couldn't synchronize partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com with partition on directory server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com.
 
 The error was:
 There are no more endpoints available from the endpoint mapper.
 
 Please verify that the address can be resolved with DNS, and that it is reachable via the transport.  If this error persists, the KCC will reconfigure the links around this server.
 The record data is the status code.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1124      Everyone      ARIEL      Internal event: The directory replication agent (DRA) failed to get a remote procedure call (RPC) binding handle for server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com, error 1753.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Schema,CN=Configuration,DC=fortdearborn,DC=com from directory ID a509f7c2-3b12-45ac-a459-b3742c2ed30d with options 55.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1060      Everyone      ARIEL      Internal event: The directory replication agent (DRA) call completed successfully.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1364      Everyone      ARIEL      Internal event: Improving the USN vector for DSA f8f05160-653c-472b-bdaf-40bacd99d315 from (9614687/OU, 9614687/PU) to (9614841/OU, 9614841/PU) for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA e378c9ff-4daf-43ef-9001-f3ac8a80df65 from USN 78208 to 78240 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA cd6fd4df-cdc8-4794-8b04-b3ca77957c50 from USN 3159835 to 3159872 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA b990a605-c780-480b-9d26-b5a3258f2ba4 from USN 193045 to 193074 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 8d203699-4ad3-41bf-b047-e3c9e32f61f4 from USN 3202027 to 3202063 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 8b082e20-fa9d-4f51-aed1-78100e7559c2 from USN 52174 to 52217 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 7708ad9e-b79a-463e-ae1c-b25804c38108 from USN 9614841 to 9614994 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 696612d6-f3fb-4451-aa9d-a7fbefcecfcd from USN 1263507 to 1263531 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 66b4d9bd-acc5-4e0e-b2b1-8c2e69705059 from USN 7950173 to 7950212 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 62d759d5-d88e-4bbe-98ee-c2950bb66cc9 from USN 2121553 to 2121598 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 2b76a2ab-ea7e-4887-bdd4-a7403f8319fe from USN 192984 to 193028 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 072b1175-2998-4c1f-a42f-55f463db4e5c from USN 799015 to 799026 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 008259f8-8ebb-4406-b31c-980c8584c0a8 from USN 6271186 to 6271233 for partition CN=Schema,CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Schema,CN=Configuration,DC=fortdearborn,DC=com from directory ID f8f05160-653c-472b-bdaf-40bacd99d315 with options 55.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1060      Everyone      ARIEL      Internal event: The directory replication agent (DRA) call completed successfully.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1364      Everyone      ARIEL      Internal event: Improving the USN vector for DSA f8f05160-653c-472b-bdaf-40bacd99d315 from (9614928/OU, 9614928/PU) to (9614989/OU, 9614989/PU) for partition CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1363      Everyone      ARIEL      Internal event: Improving the up-to-date cursor for DSA 7708ad9e-b79a-463e-ae1c-b25804c38108 from USN 9614989 to 9614994 for partition CN=Configuration,DC=fortdearborn,DC=com.
7/11/2005      8:41:12 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Configuration,DC=fortdearborn,DC=com from directory ID f8f05160-653c-472b-bdaf-40bacd99d315 with options 55.
7/11/2005      8:41:12 AM      NTDS Replication      Warning      Replication       1061      Everyone      ARIEL      Internal error: The directory replication agent (DRA) call returned error 1753.
7/11/2005      8:41:12 AM      NTDS Replication      Warning      Replication       1085      Everyone      ARIEL      Replication warning: The directory replication agent (DRA) couldn't synchronize partition CN=Configuration,DC=fortdearborn,DC=com with partition on directory server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com.
 
 The error was:
 There are no more endpoints available from the endpoint mapper.
 
 Please verify that the address can be resolved with DNS, and that it is reachable via the transport.  If this error persists, the KCC will reconfigure the links around this server.
 The record data is the status code.
7/11/2005      8:41:11 AM      NTDS Replication      Information      Replication       1124      Everyone      ARIEL      Internal event: The directory replication agent (DRA) failed to get a remote procedure call (RPC) binding handle for server a509f7c2-3b12-45ac-a459-b3742c2ed30d._msdcs.fortdearborn.com, error 1753.
7/11/2005      8:41:11 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of CN=Configuration,DC=fortdearborn,DC=com from directory ID a509f7c2-3b12-45ac-a459-b3742c2ed30d with options 55.
7/11/2005      8:41:11 AM      NTDS Replication      Information      Replication       1060      Everyone      ARIEL      Internal event: The directory replication agent (DRA) call completed successfully.
7/11/2005      8:41:11 AM      NTDS Replication      Information      Replication       1364      Everyone      ARIEL      Internal event: Improving the USN vector for DSA f8f05160-653c-472b-bdaf-40bacd99d315 from (9614978/OU, 9614978/PU) to (9614994/OU, 9614994/PU) for partition DC=fortdearborn,DC=com.
7/11/2005      8:41:11 AM      NTDS Replication      Information      Replication       1070      Everyone      ARIEL      Internal event: The directory replication agent (DRA) was asked to synchronize replica of DC=fortdearborn,DC=com from directory ID f8f05160-653c-472b-bdaf-40bacd99d315 with options 55.
7/11/2005      8:41:11 AM      NTDS SDPROP      Information      Internal Processing       1260      N/A      ARIEL      The Security Descriptor Propagator is waiting for a propagation event.
7/11/2005      8:41:11 AM      NTDS SDPROP      Information      Internal Processing       1258      Everyone      ARIEL      The directory serviced processed security descriptor propagation number 13.  1 objects were touched.
7/11/2005      8:41:11 AM      NTDS SDPROP      Information      Internal Processing       1257      Everyone      ARIEL      The directory service processing security descriptor propagation number 13 starting from node 2611.
7/11/2005      8:41:11 AM      NTDS SDPROP      Information      Internal Processing       1261      N/A      ARIEL      The Security Descriptor Propagator has been notified of waiting propagation events.
7/11/2005      8:41:11 AM      NTDS General      Information      Directory Access       1174      Everyone      ARIEL      A privileged operation (rights required = 0x) was successfully performed on object S-1-5-21-606747145-920026266-1801674531-4695.

Two things to note...one is the eventID 1174, which according to the MS knowledgebase is an error that was corrected in service pack 3...ariel is running service pack 4.

Also...the errors syncing partitions....ariel is only configured to to sync with citrixutil...which is f8f05160-.....the server that it is throwing the error for, a509f7c2-....does not seem to exist in my AD.  But even before it gets to that point...the change has already taken place.  It seems to be the events in the log snippet above starting with event ID 1174 and ending with EVENT ID 1260.

Ariel has all 5 operations masters.  Do you think if I promote a new DC, move the operations roles, and demote/blow away ariel, my problems would be solved...or does this run much deeper than just one funny server?!?

thanks again!!!!

I keep on insisting that there is refrence to an old server, which hasn't been correctly demoted ...
If you decide to investigate on this issue... http://support.microsoft.com/default.aspx?scid=kb;en-us;216498

On the other hand everythings points to this site ... you got to check everything suggested here to resolve the issue ...
http://support.microsoft.com/?kbid=839880 - it could be a big RPC issue.

I already suggsted some stuff from that site. But you are going to have to read it all up and see if you get a lead. I would love to explain it myself to ya ... but I would just complicate things.

I'll keep on looking into it. Its kinda late now. I will further analyze the log you gave me in detial tommorow, and repost.

In the meanwhile visit the link and give me your opinion

Cheers!

UICE,

OK...here is some more info based on what I am working on this AM.

1.  I belive you are right.  I belive that replication error is related to an old server...the testdc I removed earlier.  Howerver, to make life more complicated when I follow the instructions for removing it....it does not show up.  Neither does the domain it was a part of.  So it appears that some where on ariel (because I do not see these errors on other servers) it has this old testdc stuck in it.  The server was called testdc.test.fortdearborn.com.  It was the only DC in a test child domain called test.fortdearborn.com.  test.fortdearborn.com no longer exists.  I removed all entries from DNS and it appears that when I demoted it...everything was cleaned out of AD.  I have no clue where this reference actually is coming from.

2.  I ran through all of the RPC tests...checked the registry...all of the files...the only thing that failed was the gpotool.  It threw an error similar to what that article said.  Howerver none of the troubleshooting steps worked.  I downloaded that portqry tool and ran that against ariel...it was definitely listening.

3.  I ran through the same tests on citrixutil...which is the only replication partner for ariel.  during the netdiag check...I got this error...[FATAL] File \config\netlogon.dns contains invalid DNS entries. [FATAL] No DNS servers have the DNS records for this DC registered.

I'll keep digging around and post anything else I can find out.

Thank you again for all your help!!!

Find the file %Systemroot%\system32\config\netlogon.dns on the DC you moved. This file contains all the Netlogon service names the DC must register to function properly. Scan the file and find lines that begin with a semicolon, as in the following example:

;5175a911-d70b-4d3c-8df1-024ca6cd6a50._msdcs.company.net.
 600 IN CNAME server.child-domain.company.net

On the DNS server, remove all the invalid Netlogon records that conform to the format in the example and restart the DC’s Netlogon service. Restarting the Netlogon service forces the DC to reregister the Netlogon service names, which will stop the nagging messages. I haven't tested this workaround, so let me know whether it works for you.

For more information, see Microsoft article Q311354.

As for the old DC, I truly believe, becuase everything points at it (What I mean is that anyone whos have had a similar problem to yours was becuase of an incorrect, which could or couldn't be your fault, demotion of a DC ... in almost every case a test DC, but that doesn't even matter).

My point is that I really think its becuase of the old test server. Everything points at it being Ariel ... but what I believe is happening is that the Airel, which used to be a replication partner of test DC, correct?, still has an incorrect entry, for example the netlogon.dns entry, and is trying to replicate, but since it can't its "reverting" the replication ...

I couldn't specificly tell you whats going on, but the that conflict is cuasing the problems ... I am almost defineitvly positive ...

UICE,

When you spell it out that way...it most certainly seems like that is what is happening.  I will continue to pour over ariel looking for this reference.  Do you have any other suggestions on where or how I might go about finding it?

Thank you again so much for your help!!!

Ok, so from what I understand, DNS is running in Ariel Right? and in Netlogon.dns there is an incorrect entry to testDC, which is causing the trouble ... right?

By the way I extracted the info from the other reply from this site: "http://www.windowsitpro.com/Article/ArticleID/23830/23830.html"

nooo .. you are getting the netlogon.dns errors on citrixutil ..

so citrixutil was the replication partner of test DC ... hmmm ... so you would have to checkout citrixutil DC. But Ariel was the one causing trouble ...

OK ... hold on ...

I am totally getting you dizy ... Let me sort out all the information ... and I;ll get back to you with proper details of what I think is going on. Posting like this is going to drive you and me crazy...

Give me a couple of minutes ...

:)

This is getting confusing...let me try and lay things out a little neater for everyone!!!

1.  Ariel is the server that appears to be resetting the changes.  It is also the sever that generates the replication errors for what I belive was testdc (note...ariel and testdc were replication partners)

2.  Ariel passed the netdiag test, citrixutil did not...however I followed your directions on that and fixed the problem.

3.  ariel fails the gpotool test.  When I run it, I get errors when it tries to get the DC list.  The article on troubleshooting RPC erros has the exact error I am seeing.  Ariel passes all the other tests on that page.  I also went through all the troubleshooting steps and it appears everything is OK.

4.  ariel is the only server in the domain that fails the dcdiag kccevent test.

5.  testdc was demoted successfully...however I discovered a configuration error in DNS that prevented the entries from being removed.  I did that manually and everything seems to be OK in that regard.

I hope that helps clear up any confusion.

ok .. perfect ... give me a couple of minutes to get everything organized ... and I'll post up ...

more than minutes it extended to hours .. sorry bout that. I got an important phone call. Had to rush!

I'm taking a look at it now. getting every detail .. and finishing up response.

Give me a few minutes ... yes now .. for sure a few minutes ... just in case you happen to come over and read this reply ..

Try out a "netdiag /fix" on the ariel ... then run tests and see if KCC is still failing ...

I'll keep on reading ...

From the command line, type Netdiag -v or type ping -a IP_of_problem_server to make sure the host record is resolving to the correct computer.

"If you have deleted an old DC, there are a few things to check to make sure it is gone completely.

Open up the DNS Management MMC and verify the DNS records have been deleted, if not delete them.

From a command prompt run ADSIEdit.msc. Expand the Domain [DC.domain.com] container and drill down to DC=Domain, DC=COM and then expand the OU=Domain Controllers folder. If an entry exists for the old DC, delete it.

Go back up the tree to the Domain [DC.domain.com] container, and then locate the CN=System folder, expand CN=File Replication Service and then expand CN=Domain System Volume (SYSVOL) and if the old DC is still listed here, delete it.

Open up Active Directory Users and Computers and drill down to the Domain Controllers OU, if the old DC computer record still exists, delete it from here. Finally open up Active Directory Sites and Services and drill down to Sites | Default First Site Name | Servers and delete the record for the old DC if one still exists. "

--------

We got to get the old TEST server out of the way, for those replications errors to stop.

I am starting to think this could be due to DNS also. Is Ariel DNS Server too?

ahh damn ... you said you already tested netdiag ... on ariel and it all passed. There are just so many things to take into account ... I keep forgetting. In any case, you ran both those netdiag tests ... and netdiag /fix with no positive results?

OK,

Why I never noticed this before is beyond me...but for some reason (I had nothing to do with setting any of this up originally btw).  ariel has an extra suffix added to the name.

All of my dcs are computername.domain.com except ariel

Ariel is computername.xxx.domain.com

we only have one domain (domain.com).

Seeing as how Ariel is the operations master and appears to be the only one having kcc issues...I'll wait for your comments on this one.

Other than that...I did actually find a reference to a different old domain controller in the CN=Domain System Volume folder.  I removed it.  Everything else looks good as far as I can tell.


I am going to re-run all of the tests today and see if i can dig out any other old info that doesn't belong.

Thanks for your continued help with this issue.

hey ... no problem ... thats what I am here for. Lets see what the tests popup with now ...

I'll wait for that.

Did you notice the extra domain section on ariel becuase of what I suggested? just curious ...

Yes...I did.  When I went through looking for any rogue domain controllers in adsi edit, I happened to right click on Ariel's entry and in one of the windows, I noticed the path said ldap://ariel.xxx.domain.com while all the other dcs had ldap://servername.domain.com.  so I dug a little deeper and realized it was promoted to a dc with that extra .xxx in the name.  I cannot find any info online whether this actually matters...but I would guess that since xxx.domain.com doesn't actually exist anywhere...

I'll let you know what happens tomorrow...I have to build a new dc before I can remove ariel.

Thanks again for all your help!

I guess thats the best way to go ... That would in theory resolve the problem ... but its weird ... why would that be that way?

I'll await further results ...

I could seriously cry right now!

I built a new dc from scratch...redid all of my operations masters...killed ariel...changed one of the objects that had always reset itself...

about 15 minutes later...THE NEW DC CHANGED IT BACK.  The new DC (called DC1) replaces ariel, which is no longer running at all (it was properly removed from AD...I triple checked this).  

DC1 is the RID, PDC, INFRA master
Citrixutil, which is the only rep partner for DC1 is a GC, Schema Master, and Domain Naming Master

DC1 passes all of the tests we ran above.  I see nothing abnormal.  I think this is a lost cause.

If you have any further suggestions...I am willing to try almost anything.

Thanks again for all your help.

Do you think it could be a group policy issue that is causing this?

We have some basic policies...such as domain wide password policies, desktop restrictions, etc.

I seriously at this point cannot find anything wrong...netdiag, dcdiag, gpotool...all now pass with flying colors.

More fun facts.

All of the objects that keep resetting have a ghost SID in them.  What I mean by that is the SID does not seem to belong to an existing AD object.  I remove it and when the settings revert back...it shows up again.  Even more interesting...in the DS log, I get this message just as the settings are reverted:

A privileged operation (rights required = 0x) was successfully performed on object S-1-5-21-606747145-920026266-1801674531-4695.

That SID is the same SID that is assigned in the objects security tab (right click on an AD user object, go to properties, the the security tab)

Objects that don't revert do not have this SID listed in the security tab.

Then,

I get replication errors again...this time pointing to a different object than what ariel was pointing to:

event ID 1581 Failed to resolve the DNS hostname 7314acd0-8040-4950-8788-29e9ea751a7d._msdcs.fortdearborn.com into an IP address.  The error returned was: No such host is known.

event ID 1124 Internal event: The directory replication agent (DRA) failed to get a remote procedure call (RPC) binding handle for server 7314acd0-8040-4950-8788-29e9ea751a7d._msdcs.fortdearborn.com, error 8524.

event ID 1085 Replication warning: The directory replication agent (DRA) couldn't synchronize partition DC=fortdearborn,DC=com with partition on directory server 7314acd0-8040-4950-8788-29e9ea751a7d._msdcs.fortdearborn.com.
 
 The error was:
 The DSA operation is unable to proceed because of a DNS lookup failure.
 
 Please verify that the address can be resolved with DNS, and that it is reachable via the transport.  If this error persists, the KCC will reconfigure the links around this server.
 The record data is the status code.

event ID 1061 Internal error: The directory replication agent (DRA) call returned error 8524.

These events occur one right after each other at the exact time the security changes get reset.

I have no idea anymore where any of this is coming from.

I hope this helps you.  I am beginning to think I am screwed no matter what.

Even more info...

My servers are trying to replicated with a retired partner.  I can see it if i go to the active directory replication monitor and choose show retired replication partners under the option screen.

So that is the cause of the replication issue...and the server from the above post (7314acd0-8040....) is what used to be ariel.  I have run through the metadata cleanup and there is no reference to ariel anywhere I can find.

So...one mystery half solved...if I can get rid of that...all will be good.

now...if I can just figure out what that damn SID is or how to remove it...

I;ve been kinda busy!

Sorry .. I'll post up on this in a few hours ... I think I know what to do next ...

Hold up for me ... don;t cry just yet ... hehehe

Hey

No problem.  In the mean time...I'll fill you in on what I did last night.

1.  I reset the default domain policy and default domain controllers policy via the ms knowledgebase (226243 & 267553)

2.  I reviewed all of the group policies and removed anything that had control over the OUs that contained the bad users.

3.  I recreated all of the above GPO and propagated them out.

4.  I tried resetting one of the accounts...no luck.

I'll hold back my tears until I see your post this afternoon!

Thanks again!

SORRYYYY very busy mate .. I haven;t forgotten about you.

I promise I'll post ASAP!!

sorry seriously ...

You must think I forgot about you man. BELIEVE ME .. I have not!

I am just running into all these meetings my company has poped up recently with no notice. I am leaving all my question like way open. All these people are hanging in there just like you ....

Terribly sorry mate on this ... really!

As soon as I get a little more than a coffee break (like now which is why I am writing tihs to you) I'll get back on track.

Please don't get mad...

Any posible updates on your situation? I bet you really need this fixed and pronto ... they got to be stomping on you hard ...

sorry mate .. really!

I'll see if I can get to this tommorow afternoon...

Best of luck in the meantime,

UICE

UICE,

Don't worry about it.  I am not mad at all.  While i would like to get this solved...at this point in time it is more of an annoyance than an actual problem.  I keep plugging away...but I haven't gotten anywhere.

I still have my servers that are trying to sync with retired replication partners and I still have that mystery SID that appears to be attached to all the objects that reset themselves.  I now belive that the cause is the infrastructure master role...but that is more of guess based on the fact that I don't think the PDC or RID masters would do such a thing.  I am going to be out all today...so i won't have time to get to anything until tomorrow.

I really appreciate all your help...even if takes until next month to solve it...if we get it eventually...that is all I am hoping for.

Hope things get easier for you...I'll post again if I discover anything between now and tomorrow.

Thanks again!!!

OK...here is something more for you to think about.  I don't think my Infrastructure master is/was deleting tombstone objects.  I looked at the following MS knowledgebase articles (265090,248047,258310) and realized that the attribute value described was not set and the registry key mentioned was not there either.  I added both (I hope this make sense) and bounced my infra master and will wait and see what happened.

When I used ldp.exe I found at least 200 deleted objects that had not been removed.  I don't really know why this is...but hopefully I have taken steps to fix it.

The more we get into this...the more I realize how messed up this entire AD is.  I wish I could do a format c: on the entire thing and start over.  This is a mess!!!

I'll keep digging and post any other oddities I find.

Thanks again for all your help.

I was exactly going to suggest something like this
tombstoning objects aren't being correctly deleted ... you are going the right way mate.

Let see what happens.

Me in the meanwhile have to continue with meetings ... important ones too. Alot of people are getting laid off so I got to do my best ...

I'll keep on stoping by as much as I can ... but can't promise much ...

Cheers!

UICE,

Sorry to hear about all the troubles.  I hope things work out OK.

As for me...same problem.  I have looked around for more info on deleted objects...apparently I did something good because I am down to 21 objects when I run the query from one of the MSKB articles above.  However my issue still exists.  The nTSecurityDescripter continues to get reset.

If you (OR ANYONE ELSE) has any further ideas...please let me know.  Otherwise I fear we may have exhausted all current options.

OK,

Well things seem to be working properly now in terms of garbage collection.  I am down to 16 objects...though I still think I may have a couple stuck in there.  The security being reset on my objects is still happening.  At this point...I have no idea what else to do.

If anybody else has any thoughts...please post them.

Thank you.

sit tight man ... almost done with things at work ... they are starting to give me a brake. This weeked i'll try to post up .. ok?

cheers!

Did you run steps 8 down from http://support.microsoft.com/?kbid=265090 ?

You did try everything out here right? http://support.microsoft.com/default.aspx?scid=kb;en-us;216498 ?

Mate ... I have read everything there is to read basiclly on this issue. All I can say is that you should check that the tombstone life if set to 2 ... as in 2 days ... as in step 8 from that article you gave me ...

And then try using ASDIedit again from the second link I gave you.

Hey man,

I have done everything on those articles.  I am 99% certain that garbage collection is working properly now.

I have one last piece of information...I confirmed this last night and this morning by moving them around...

The Domain Controller that holds the PDC operations master role is the one responsible for the resets...always.  I moved the roles around last night and changed the security permissions.  This morning, I ran a repadmin /showmeta on the object and the server that had the PDC was listed as changing the Security Descriptor.  So I moved the PDC role to another server and tried again.  Sure enough about one later...that server (the one with the PDC op master role) was listed as resetting the object.

So that is my problem.  The PDC operations master is resetting the security descriptor bit on some of my AD objects after I make change.

If this rings a bell or you have any additional ideas on how to proceed...please let me know.

Thank you again so much for all your help.

Sorry my Friend ... Work has been terrible latley. We are going through a sort of crisis. I will try to post back up as soon as posible. But I really can't promise much.

Terribly sorry!

Hey UICE,

You have been incredibly helpful.  I fully belive my AD is far better off than it ever used to be.  Unfortunately...everything we have done...while completely beneficial...has failed to solve the original problem.

If we are out of ideas...perhaps I can somehow give you points for all the other problems you solved and re-ask this question?

Not sure what else to do here.

Thanks again sooo much for everything...the best of luck with anything!!!

Does anyone else have any suggestions or need additional info?  Or is this a dead end?

Thanks!!!

Mate,

I;ve been so busy I haven;t even been on Experts Exchange for the past month, which as a expert is really bad.... I lose alot of points for not being able to keep up as a EE expert ...

Not only that, but I feel like I kinda let you down. I'll try to post up as soon as I can. But I am so busy.... I just can;t let you know when. Work, Family ... studies ... its just all too much, and I felt like I had to just let go for a while ...

I'll try to post something up tommorow, or the next day. But I am sooo busy ..

Sorry mate, really ...

UICE

What is the exact problem? Describe in short and simple. So Its easy for me to troubleshoot. Going through this thread is a huge pain.

Thanks
SystmProg

SystmProg,

Here is the issue in short, sweet detail.

I have a few AD objects (users and groups) that somehow lost the default permissions (go to the object, click on the security tab) such that account operators no longer have access to them.  So I go into the object, click on the security tab, and check the box to enable inheritable permissions.  The permissions reset...are correct...and propagate out.  A short time later (sometimes 15 minutes...sometimes one hour) the Domain Controller which holds the PDC role removes the changes I made to the security permissions.

I see this by running a repadmin /showmeta on the object and looking at the nTSecurityDescripter property.

The domain is Win2k and all DCs have Service Pack 4.  I cannot tell you exactly when this started...but it has been happening for a while.  There appears to be no common link between the objects that show this issue (they are in different OUs, different security groups, etc...)

If there is any more info you need...please let me know and I'll get right back to you.

Thank you much!!!!

Enable Active Directory Auditing on the objects and then find the culprit.

Does it make sense?

OK...sorry for the month delay.  A phone system implementation and vacation pulled me away from this for about three weeks.  i tried enabling auditing on the one of the objects that keeps getting reset and it really didn't tell me anything.  I set it up to audit access by dc1 (the domain controller that keeps resetting the permissions) and I don't see anything.  Perhaps I am not setting it up right?  Or perhaps I am just missing something.  Could you elaborate a little more on how I should set this up and what exactly I am looking for?

Thank you.

Is there anybody else who can help me?  I am trying SystmProg's suggestion but do not think I am doing it correctly.  What type of audit access do I need in my domain and what/who should I be looking for on the object itself?

OK...I finally figured this one out.  The issue was the SDPROP and AdminSDholder removing permission from accounts that currently are...or used to be...part of a protected group.  Apparently in Service Pack 4 they added a number of groups to the protected group list.  There are a number of MS knowledgebase articles dealing with this issue.  For me...the solution was to remove the accounts from the Account Operators group and just use delegated permissions.  I think I managed to break something else...so I am going to post that as another question.

Thank you to UICE for trying so hard.  I cannot say enough for all the advice he gave me.  Even though he did not solve this particular problem, I learned a lot by reading his posts and following his advice.  Invaluable if you ask me.  Thanks again.

COULD YOU PLEASE REFUND POINTS BUT NOT DELETE THE POST. I WOULD LIKE TO HAVE THIS POST OPEN FOR FUTURE REFRENCE.

kfrankovich , I am so terribly sorry I could not post up here. I feel ashamed ... I will explain why I haven't been posting here at EE, though....

I have been hired by DestiNY USA ( http://www.destinyusa.com ) - we have just been so busy with all the process. Thousand of things to do. Its such a HUGE project!

I am really sorry mate. I am glad you could figure it out though! - thats what its all about, and it proves just how professional you are mate! - I really congrat you on that!

Hope we keep in touch.

Cheers,

UICE

I agree.

UICE,

Don't worry about it.  Even though you were unable to solve the issue I was having, you actually helped me fix issues I had and didn't even know about.  My active directory is a lot better off because of all the advice and tips you provided.  I think there is some great information in here and I agree with you and SystmProg and hope that they will leave this up as a reference for others.

Good luck with your new position...I hope it works out for you.

Thanks again for everything!!!

kfrankovich, congrats to you again my friend ...

And SystmProg, my old friend, get ready cuase I am planning on coming back soon :-P

Cheers guys! :-D

UICE

*grin*

Cheers