Solved

All GPO with GUIDS beginning with numbers are deleting themselves 2012 R2

Posted on 2016-09-29
14
39 Views
Last Modified: 2016-11-04
Hello
This seems like the deepest type of SYSVOL corruption that I've encountered.  I've been working with a large client on replication issues in a 2012 R2 - only environment.  

I did a non-authoratative restore from PDC to DC2 and DC3 at remote sites on Tuesday night.  Everything seemed to proceed perfectly that night and throughout wednesday and thursday morning.  However today (Thursday afternoon) someone noticed that all of the GPOs that began with NUMBERS in the GUIDS disappeared from PDC Only.  This had to have happened today.  The ones that start with letters for example {CE31B9E2-D06E-420B-97E6-4A44C62C98A8} are all still present.  The ones that start with numbers are Gone.  

this leaves 26 of the original 88.  I was able to pull 85 from DC3 or DC2 and says you require permission to copy the last three GPOs "Please contact your administrator." and as you probably already figured,  I am administrator.

I tried manually copying the policies back to sysvol and it quickly re-deleted them.

What could be happening here?
0
Comment
Question by:csg-unit
  • 7
  • 6
14 Comments
 
LVL 6

Accepted Solution

by:
No More earned 500 total points
ID: 41822223
1, Did you run DCdiag or Repadmin  ?

2, I would personally do authoritative restore

3, Check PDC role server Hard drive if point 1 show no errors

4, Do you have policy definitions in central store ?

Also performance counters for replication could show whats actually happening , if you set it up

optionally : Move PDC role to another server
0
 
LVL 1

Author Comment

by:csg-unit
ID: 41822244
1.  All DCDIAG tests pass except:  

      Starting test: Services

         Could not open Remote ipc to [PDC.Mydomain.Local]: error 0x4b8

         "An extended error has occurred."

         ......................... PDC failed test Services

      Starting test: SystemLog
         An error event occurred.  EventID: 0x00000422
            Time Generated: 09/29/2016   12:58:15
            Event String:
            The processing of Group Policy failed. Windows attempted to read the file \\Mydomain.local\sysvol\Mydomain.local\Policies\{31B2F340-016D-11D2-945F-00C04FB984F9}\gpt.ini from a domain controller and was not successful. Group Policy settings may not be applied until this event is resolved. This issue may be transient and could be caused by one or more of the following:  [AND IT REPEATS]


An error event occurred.  EventID: 0x00000422

            Time Generated: 09/29/2016   12:59:26

            Event String:


            The processing of Group Policy failed. Windows attempted to read the file \ (same as above message)
etc.etc..

etc.etc..


2. I  can still do an authoratative restore from DC3 but it crushes the bandwidth for 2 hours and must be done tonight, but this has to be fixed by morning (I may open a microsoft professional services ticket)

3. What do you mean by the PDC Role Hard drive?  It is a VM using a SAN in hyper-v.  the SAN is showing good, but I suppose I can run a chkdsk afterhours, but it is almost too time consuming to think about right now.

4. the central store shows:
\\Mydomain.local\sysvol\Mydomain.local\Policies\    26 policies NOT 88



The other DCs are OK
0
 
LVL 6

Expert Comment

by:No More
ID: 41822259
3, What i meant is to check HDD on server which is PDC -But if you have SAN don't bother

IPC or I know it as share IPC$ remote IPC - Bad sign
(Inter-Process Communication (IPC$) is used for data sharing between applications and computers)

Seize PDC  FSMO role by different server ASAP!!

You might also need to remove ADDS role on that server with current PDC and then clean up metada in AD and reinstall ADDS on that server
0
 
LVL 1

Author Comment

by:csg-unit
ID: 41822328
I changed FSMO role holder to DC3
as for the last comment that sounds easier said than done.  I'll take a look, thanks.  in the mean time I am still open to suggestions.
0
 
LVL 1

Author Comment

by:csg-unit
ID: 41822332
Additionally I want to point out that this client for some reason is still using FRS for SYSVOL Replication instead of DFSR despite being an all 2012 R2 environment (DCs, still some '08 members) in a 2008 forest level.
0
 
LVL 1

Author Comment

by:csg-unit
ID: 41822350
Repadmin: running command /showrepl against full DC localhost

Default-First-Site-Name\PDC

DSA Options: IS_GC

Site Options: (none)

DSA object GUID: 0022f952-3ecb-4caf-a754-dfab65fe93ad

DSA invocationID: 8463c260-9822-4824-8f82-86249f9f00a8



==== INBOUND NEIGHBORS ======================================



DC=MyDomain,DC=local

    Hotsite\DC3 via RPC

        DSA object GUID: 504c1d7a-fcc0-41c6-8e88-45f5dc2b4f63

        Last attempt @ 2016-09-29 14:52:39 was successful.

    Default-First-Site-Name\DC2 via RPC

        DSA object GUID: 088c6fd7-1fd7-420b-ac7c-2013edb6a255

        Last attempt @ 2016-09-29 15:04:42 was successful.



CN=Configuration,DC=MyDomain,DC=local

    Hotsite\DC3 via RPC

        DSA object GUID: 504c1d7a-fcc0-41c6-8e88-45f5dc2b4f63

        Last attempt @ 2016-09-29 14:52:39 was successful.

    Default-First-Site-Name\DC2 via RPC

        DSA object GUID: 088c6fd7-1fd7-420b-ac7c-2013edb6a255

        Last attempt @ 2016-09-29 14:55:32 was successful.



CN=Schema,CN=Configuration,DC=MyDomain,DC=local

    Hotsite\DC3 via RPC

        DSA object GUID: 504c1d7a-fcc0-41c6-8e88-45f5dc2b4f63

        Last attempt @ 2016-09-29 14:52:39 was successful.

    Default-First-Site-Name\DC2 via RPC

        DSA object GUID: 088c6fd7-1fd7-420b-ac7c-2013edb6a255

        Last attempt @ 2016-09-29 14:55:32 was successful.



DC=DomainDnsZones,DC=MyDomain,DC=local

    Hotsite\DC3 via RPC

        DSA object GUID: 504c1d7a-fcc0-41c6-8e88-45f5dc2b4f63

        Last attempt @ 2016-09-29 14:52:39 was successful.

    Default-First-Site-Name\DC2 via RPC

        DSA object GUID: 088c6fd7-1fd7-420b-ac7c-2013edb6a255

        Last attempt @ 2016-09-29 14:55:32 was successful.



DC=ForestDnsZones,DC=MyDomain,DC=local

    Hotsite\DC3 via RPC

        DSA object GUID: 504c1d7a-fcc0-41c6-8e88-45f5dc2b4f63

        Last attempt @ 2016-09-29 14:52:39 was successful.

    Default-First-Site-Name\DC2 via RPC

        DSA object GUID: 088c6fd7-1fd7-420b-ac7c-2013edb6a255

        Last attempt @ 2016-09-29 14:55:32 was successful.
0
 
LVL 6

Expert Comment

by:No More
ID: 41822357
Well sometimes it's hard to give guys help here, as I can't see diagram of the infrastructure you dealing with

But well, i would rather remove those 2008 dc, too old for year 2016
0
 
LVL 6

Expert Comment

by:No More
ID: 41822361
Do DCdiag aswell
0
 
LVL 1

Author Comment

by:csg-unit
ID: 41822369
Three remote sites, each has a DC.   site 1 was PDC, site 3 is DC3, site 2 is DC2.  I changed fsmo role holder from pdc to dc3.   They each have about 100 computers each.    there are only 2008 member servers that have no role in this issue.
0
 
LVL 6

Expert Comment

by:No More
ID: 41822386
OK, Now i got it, not easy to reinstall ADDS, but you could create new VM on Hyper-v
0
 
LVL 1

Assisted Solution

by:csg-unit
csg-unit earned 0 total points
ID: 41822401
yeah we may leverage that if it comes to it.  I manually copied the policies back to PDC(Now a regular DC, DC1).  after 10 minutes or so they disappeared again.

I plan to do an authoritative restore tonight from DC3.

I did a health check for a migration to DFSR according to
https://blogs.technet.microsoft.com/filecab/2014/06/25/streamlined-migration-of-frs-to-dfsr-sysvol/

and everything came back normal/good to go.
0
 
LVL 6

Expert Comment

by:No More
ID: 41822421
Well, i would suggest one more thing , Backup polices using group policy management console and import it do dc same way
0
 
LVL 1

Author Closing Comment

by:csg-unit
ID: 41874701
This was ultimately fixed by wrestling with permissions on some locked files for the entire night in the SYSVOL, performing many authoritative restores to force propagation until everything was in place, and while I had the chance I quickly migrated to DFSR from FRS and confirmed all was working.  I also had to delete some 700MB of msi files that had no function in the sysvol share before all this began, which helped propagation immensely, which goes without saying.

This was a huge step in the evolution of the networks' health.  I suggest migrating to DFSR to everyone with a modern environment.
0

Join & Write a Comment

Suggested Solutions

Every now and then, Microsoft does something that totally impresses me. It doesn't happen often, but in this case I must say I am thoroughly impressed with Windows Server Backup. One of the long time issues with Windows Backup has been the ability t…
Synchronize a new Active Directory domain with an existing Office 365 tenant
In this Micro Tutorial viewers will learn how they can get their files copied out from their unbootable system without need to use recovery services. As an example non-bootable Windows 2012R2 installation is used which has boot problems.
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now