Backup Domain Controller Failed to self Promote

Hello all,

I could really use some assistance with this situation as I am not sure how to proceed and I do not want to make things worse.

This is actually a pretty simple setup……

One Domain with just two servers…. a Windows 2003 server (Dellsvr1) which was the Primary Domain Controller and a Windows 2008 server (Dellsvr2) (with service pack 2) which was setup as the Backup Domain Controller.

The Windows 2003 server (Dellsvr1) crashed because TWO of the eight hard drives failed. It was setup as a RAID 5. I have file by file backups and I also made images of it. So, I was able to get this company going again by installing their database apps onto another system for right now. But I can not get either image to load …. And that is a long story itself.

But where I am at right now is the Windows 2008 Backup Domain Controller (Dellsvr2) failed to promote itself properly when the Primary Domain Controller died.

And I am stuck with an Event ID error of 1864. I just do not want to make matters worse so I thought I would ask for advice before proceeding.

Please see the attached PDF screen shots of DCDIAG from the Windows 2008 Backup Domain Controller (Dellsvr2).
Will LipscombeOwnerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Shaun VermaakTechnical Specialist/DeveloperCommented:
If Dellsvr2 was never promoted to a DC successfully, you might need a specialist to fix this for you. I suggest opening a gig and I recommend you do not blindly follow troubleshooting steps without a backup of the current state.
0
ITguy565Commented:
Please run DCDIAG /v on the DC and post the results to this forum. If I had to guess right now the server that went down carried the vast majority of your fSMO roles or all of them. If those roles are not available the server will not function as a DC.
0
Seth SimmonsSr. Systems AdministratorCommented:
...which was setup as the Backup Domain Controller.

backup domain controllers haven't existed since NT and went away with windows 2000
one domain controller has the PDC emulator FSMO role for compatibility reasons but all domain controllers are otherwise equal in terms of login process, etc.

the Windows 2008 Backup Domain Controller (Dellsvr2) failed to promote itself properly when the Primary Domain Controller died.

a secondary domain controller won't 'promote itself' when another fails.  you need to seize FSMO roles on another domain controller (the 2008 server in this case) and cleanly remove the dead server from AD

Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller
https://support.microsoft.com/en-us/help/255504/using-ntdsutil-exe-to-transfer-or-seize-fsmo-roles-to-a-domain-control

Step-By-Step: Removing A Domain Controller Server Manually
https://blogs.technet.microsoft.com/canitpro/2016/02/17/step-by-step-removing-a-domain-controller-server-manually/

Please see the attached PDF screen shots of DCDIAG...

there is nothing attached; not worth looking into that at this point with a dead domain controller and only 1 left

The Windows 2003 server (Dellsvr1) crashed because TWO of the eight hard drives failed. I

then turn it off and leave it off
2 drive failures in a RAID 5 and you lose the array

bottom line - seize the roles, remove the dead DC from AD, build another domain controller for redundancy (not windows 2003)
you have some work to do
4

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

ITguy565Commented:
Glad to see that you agree with me Seth :P

I would still like to see the DCDIAG report even though I am 90% sure of what has happened.
0
ITguy565Commented:
@ Will
Keep in mind, if you resort to "seizing the FSMO roles" from another DC, you can "NEVER" reintroduce that DC back into the environment. You must clean install that box. DO NOT attempt to bring it back you will have conflicting roles and seriously jeopardize the integrity of your Domain.
0
ITguy565Commented:
@Will,

Do you know if the second server was a Global Catalog server? We know the failed one was.
0
R@f@r P@NC3RVirtualization SpecialistCommented:
Good afternoon,

I agree with John Kilgore and Seth Simmons, if you have many doubts, I invite you to open a report with Microsoft and they will advise you for as long as necessary and they will make all kinds of suggestions.

Regards...
0
Will LipscombeOwnerAuthor Commented:
First…. thank you all for taking the time to assist me with this issue (smile).
      --------------------------------------------

@Shaun Vermaak

“If Dellsvr2 was never promoted to a DC successfully, you might need a specialist to fix this for you. I suggest opening a gig and I recommend you do not blindly follow troubleshooting steps without a backup of the current state.”

Unfortunately, this company that I support cannot afford that…. but thanks for the suggestion.

      ----------------------------------------------

@John Kilgore

“Please run DCDIAG /v on the DC and post the results to this forum. If I had to guess right now the server that went down carried the vast majority of your fSMO roles or all of them. If those roles are not available the server will not function as a DC.”

I absolutely positively DID attach a PDF which showed a screen shot of the DCDIAG results from Dellsvr2 to my original post. Why it did not show up on your end…. I have no idea. I will resubmit both a Word Doc and a PDF containing the DCDIAG results to this reply. But non-Verbose mode.

      --------------------------------

@Seth Simmons

“backup domain controllers haven't existed since NT and went away with windows 2000
one domain controller has the PDC emulator FSMO role for compatibility reasons but all domain controllers are otherwise equal in terms of login process, etc.”

 Ooooooook……Please forgive me but I’m not quite sure why you are saying that Backup domain controllers have not existed since NT…. When I originally setup this Power Edge 2900 in 2009 one of the options was specifically to set it up as a Backup Domain Controller……. And as of about three years ago…. I even tested it. I completely shut off the Primary DC and I saw that the Backup domain controller (Dellsvr2) took over the domain and allowed everyone to log on as usual. And at that time Server Manager did NOT report any problems.

Something happened SINCE my tests…… and I am not sure what.

When I originally setup Dellsvr2 I remember that I even had to update specific files on it in order to get it to work correctly as a Backup Domain controller. Its been several years…. I can look up the procedure and related Microsoft KB’s about that if you want me to.

“a secondary domain controller won't 'promote itself' when another fails.  you need to seize FSMO roles on another domain controller (the 2008 server in this case) and cleanly remove the dead server from AD”

Sorry again…. But as I mentioned above I did actually test that Dellsvr2 was working as a Backup Domain Controller.

“there is nothing attached; not worth looking into that at this point with a dead domain controller and only 1 left”

As I mentioned to John above…. I absolutely positively DID attach a PDF containing screen shots of DCDIAG from Dellsvr2 to my original post.

On MY end…. it said that it uploaded correctly. I will attach again and hopefully it will make it to your end this time.

“then turn it off and leave it off
2 drive failures in a RAID 5 and you lose the array”

Yes sir…. That is absolutely correct…. Two of the eight drives dyeing killed the array.

You know…. on that side of the issue…. I always have file by file backups of ALL the systems that I support… but in the case of this old Dell Power Edge 4400… I knew it was not going to last much longer…. In fact, the ONLY reason that it was still running is because this company had two ancient database apps that they said they could not afford to update. And those outdated apps would NOT run on any OS newer then Windows Server 2003. I told them that they needed to either update those two apps or move all of the data to newer better apps. But they told me they could not afford it. So, I kept this old Dellsvr1 which was originally a Windows 2000 server (upgraded to a 2003 server by someone else) going for as long as I could.

I even made TWO images of it…. one using the Windows Server built in backup… I even made the floppy disks that it requires.
And I also made a second image using the free version of Drive Image XML.

So, when the old Dell Power Edge 4400 finally died (The Primary Domain Controller) …. My first priority was to get this company working again. At this time, they had no choice but to buy updated versions of their database apps…. I guess they made arrangements to make payments or something??? So, I used another system for the database apps.

Anyway…. after I got them up and running again…. I turned my attention to the Power Edge 4400. Fortunately, I found two hard drives on line just like the ones that had died…. And since I had TWO images I thought that I would repair the RAID 5 array and put one of the images on it.

So…. First, I installed the two new drives and formatted and tested ALL eight of the 36 Gigabyte hard drives. So currently the RAID array is working just fine.

I went to run the Dell Open Manage Server Assistant CD (amazingly this company still had the optical disks for this server) …. So, I could install Windows server 2003 and then put the image back on it…. But the Open Manage Server Assistant CD freezes at the language selection screen every single time and I can’t get past that point…. No matter what I do.

So, then I tried the Drive Image XML …… but the Knoppix boot CD also stalls before getting me to a desktop so I can install that image.

That’s why I am now trying to see what can be done to correct the issues with Active Directory on Dellsvr2 (the Dell Power Edge 2900).

Anyway…. Currently …. yes, the OLD Dell server is OFF and just sitting here.

“bottom line - seize the roles, remove the dead DC from AD, build another domain controller for redundancy (not windows 2003)”

Yes…. This sounds like what I will have to do…. But I do not understand how I can seize the roles from the Primary Domain Controller (Dellsvr1) when that server is no longer available??

Anyway, I will follow the links that you sent me and run through that process and see how far I get.

Thank you VERY much for getting those links for me.

      --------------------------------------------

@John Kilgore

“I would still like to see the DCDIAG report even though I am 90% sure of what has happened.”

 I will reattach those screen shots.

“Keep in mind, if you resort to "seizing the FSMO roles" from another DC, you can "NEVER" reintroduce that DC back into the environment. You must clean install that box. DO NOT attempt to bring it back you will have conflicting roles and seriously jeopardize the integrity of your Domain.”

Yes…. Thank you… I understand that.

“Do you know if the second server was a Global Catalog server? We know the failed one was.”

I followed this procedure:

To test whether a domain controller is also a global catalog server:
1.      Click Start, point to Programs, point to Administrative Tools, and then click Active Directory Sites and Services.
2.      Double-click Sites in the left pane, and then locate the appropriate site or click Default-first-site-name if no other sites are available.
3.      Open the Servers folder, and then click the domain controller.
4.      In the domain controller's folder, double-click NTDS Settings.
5.      On the Action menu, click Properties.
6.      On the General tab, view the Global Catalog check box to see if it is selected.

And…. YES…. there IS a check mark next to “Global Catalog” for Dellsvr2.

I will also include a screen shot of this. Hopefully it makes it to you.

      ------------------------------------

Anyway…. I wanted to get back to you guys as quickly as possible…. But obviously this is still a work in progress.

One other thing…… I am unsure of the results of going past the “Tomb Stone Lifetime” and I don’t want matters to get any worse…. so, I increased the Tomb Stone Lifetime to six months for right now.

I just want to thank everyone once again for their assistance with this issue. (smile)
DCDIAG-results-from-Dellsvr2_031918.doc
DCDIAG-results-from-Dellsvr2_031918.pdf
Global-Catalog-results-from-Dellsvr2.doc
Global-Catalog-results-from-Dellsvr2.pdf
0
ITguy565Commented:
OK, so thanks for the screen shots. This confirms what I originally thought. The FSMO roles were all Held by DellSvr1

At this point you will need to do the following:

Quoting Seth's Processes above :
a secondary domain controller won't 'promote itself' when another fails.  you need to seize FSMO roles on another domain controller (the 2008 server in this case) and cleanly remove the dead server from AD

Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller
https://support.microsoft.com/en-us/help/255504/using-ntdsutil-exe-to-transfer-or-seize-fsmo-roles-to-a-domain-control

Step-By-Step: Removing A Domain Controller Server Manually
https://blogs.technet.microsoft.com/canitpro/2016/02/17/step-by-step-removing-a-domain-controller-server-manually/


This is the ONLY way to clear AD and get it back in working order.
Yes…. This sounds like what I will have to do…. But I do not understand how I can seize the roles from the Primary Domain Controller (Dellsvr1) when that server is no longer available??

To answer this question, If dellsvr1 was online the all you would need to do is "transfer the FSMO roles to another server" In the event the server is no longer reachable the "Seize" option was presented to the administrators.

This is why you can no longer introduce this server back into the environment. If you were to do so, you would have two servers running the same FSMO roles which can not coexist on the same network.

Please use the NTDSUTIL command in order to clean-up and seize the roles of the DELLSVR1 which is no longer available.

On another note:

Try https://www.groovypost.com/howto/bios-uefi-setup-guide-boot-from-cd-dvd-usb-drive-sd-card/ 

I believe your UEFI settings  in the BIOS could be preventing your boot to CD's and Removable media.
0
ITguy565Commented:
In reality, it should only take you less than an hour to restore Connectivity to the domain following the procedure above.
0
Seth SimmonsSr. Systems AdministratorCommented:
Please forgive me but I’m not quite sure why you are saying that Backup domain controllers have not existed since NT

with NT, a domain controller was either a PDC or BDC
a PDC  was read/write for updating user accounts, adding users and computers on the domain, or any other domain changes
a BDC had a read-only copy of what the PDC had...it could authenticate user logins and anything else that it didn't need to write
if your PDC was down, you were hurting.  if the PDC died, you did have the ability to make a BDC a PDC but my memories beyond that are vague since i haven't worked with NT in at least 15+ years

the term 'backup domain controller' today is just an additional domain controller where they are all read/write though only some functionality is lost when the server with the FSMO roles dies.  users can still login but certain domain functions won't work if, say, the RID master can't be contacted you won't be able to create user/computer objects since it needs the RID master to assign a SID to the object.  another domain controller won't take over those roles unless you transfer (or in your case) seize those roles

good that you have backup images of that DC to restore other files from - just don't use a DS restore to try restoring it as you will probably cause more AD issues
1
ITguy565Commented:
@Will

1. Please give us a status when you get a moment with the Domain Controller.


In reality, if I were in your situation, I would just seize the roles that were on DellSvr1 decommission the box, and pick up any remaining pieces with software!

There would be far less time wasted if this were done. (Probably 1Hr in work) to restore connectivity. Then you would need to concentrate on any other roles that server might have had such as custom software, print server, remote Access, or whatever else that server may or may not have been doing.
0
Will LipscombeOwnerAuthor Commented:
@John Kilgore and Seth Simmons

Thank you very much for your reply and for explaining that.

I will seize the FSMO roles onto Dellsvr2 and then remove Dellsvr1 from Active Directory as soon as possible ..... possibly this weekend when (hopefully) no one is on the Domain.

I will do the above using the links that you guys sent to me.

————

And about the optical disks not booting.....

I know it must sound like I am idiot ..... but I have been supporting computers since 1988 when I was top of my class of fifteen students at college.... straight A.

I know what a BIOS (Basic input output system) is..... and also what the newer UEFI (Unified Extensible Firmware Interface) is all about.

I used to configure boot disks all the time.

So..... perhaps I did not explain myself correctly.....

The Server Assistant CD DOES in fact boot just like it should.... and it is the correct CD for this Power Edge 4400.... with no scratches on it..... but it stalls ..... just sits there with no mouse curser at the Language Selection Screen. This Power Edge 4400 still has its original PS2 keyboard .... but a few years ago I had to retire the old PS2 ball type mouse because it was getting too difficult to use. So I am now using an optical USB mouse. I am wondering if the Boot CD needs to see a PS2 mouse?

Please see the attached pics of the Server assistant CD that I am using AND exactly where it is stalling.

There are no add in cards in this system.

According to what I have read in the Dell support forums. There may be an issue with this version of the Dell Server Assistant CD for this model Power Edge that no one has ever corrected??

Anyway..... I wanted to reply back to you guys as quickly as possible..... but obviously this is still a work in progress.
Server-assistant-CD-screen-shot-0322.jpg
Where-it-stalls-032218.jpg
0
ITguy565Commented:
Thanks for the reply, I will wait to hear back from you next week. Have a good weekend!
0
Will LipscombeOwnerAuthor Commented:
This post is NOT abandoned...... I can only go so fast.......Because this is a REAL server on an Active Directory domain .... I FIRST had to make an image of the hard drives......then I have to schedule a restart with the end users.

For the image: This time I down loaded, installed and configured a fully functional 30 day trial version of Acronis for servers. And then I made an image of this Dell Power Edge 2900.(smile)

I will keep you all informed and Thank You.
1
Will LipscombeOwnerAuthor Commented:
Hello all,

Well I finally had a chance to work on this Power Edge 2900 (Dellsvr2) this weekend.

During this last week I made a valid image using the newest Acronis True Image for servers and I validated it. And of course I also have file by file backups.

I followed both of the links that Seth provided and ITguy565 verified (Thank You, Thank You) and although those OTHER web sites had some errors (basically the syntax was NOT 100 percent accurate) I was able to compensate and seize the FSMO roles and then remove the residue of the dead server from this domain.

So far it looks GREAT...... but I need to keep an eye on it for a few days. But so far server manager and event viewer look MUCH better.

Consequently I am considering purchasing my own subscription to Experts Exchange.

I still need to verify that this Domain Controller is now fully in charge of this domain by updating the end user passwords.

I’ll be back (smile).
0
Will LipscombeOwnerAuthor Commented:
Seth Simmons was RIGHT on and ITguy565 verified.
I am purchasing a subscription right now.
Thank you VERY much (smile)
0
ITguy565Commented:
glad we could help!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Server OS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.