Solved

Many services fail to start on new 2008 DC- Access Denied

Posted on 2010-09-22
29
3,777 Views
Last Modified: 2012-12-14
Hi,

I built 3 new 2008 standard domain controllers from scratch. All are virtual machines on VMWare. On each server after adding the DC role, dcpromo, etc. and then running Windows updates a bunch of services fail to start due to access denied.

Failing services:
Base Filtering Engine (BFE)
IPSec Policy Agent
Windows Time
IKE and AuthIP IPSec Keying Modules
DHCP Client
Distributed Transaction Coordinator (DTC)
Diagnostic Policy
Network List Service
Network Location Awareness
Windows Firewall

I had an email ticket with Microsoft on this and wasted 2 weeks of time. They kept focusing on the permissions on the BFE service. The neccessary permissions that were NOT in place were for the NETWORK SERVICE and LOCAL SERVICE. I eventually found a link to an article on this very problem. http://techruckus.com/forum/vista-dhcp-base-filtering-service-access-denied-t-t91.html

I followed the instructions and it fixed the problem. Basically, you install the SubInACL and run a script which recreates all the permissions and then the services start.

Now, the problem....this problem has reappeared on each of these 3 DCs. I've had to rerun this script now a second time on each DC. I don't think it is a group policy problem because after a reboot all the services still are ok. It's at some point a few days later that the services fail or stop and can't start because these NETWORK SERVICE LOCAL SERVICE accounts don't have access. The accounts just aren't even there. So, I have to rerun the script and that fixes it.

I'm very concerned about this as when these services are stopped it causes problems in DC replication and Exchange!

Any thoughts on why these permissions keep being removed?
0
Comment
Question by:scubadiver_dave
  • 13
  • 8
  • 5
  • +2
29 Comments
 
LVL 11

Expert Comment

by:willettmeister
Comment Utility
Do you have any other services on you network that woudl monitor accounts and automatically change/update them.  I know some security programs do that.  

0
 

Author Comment

by:scubadiver_dave
Comment Utility
Not that I can think of. I don't even have AV on these boxes yet.
0
 
LVL 6

Expert Comment

by:SHIELD1
Comment Utility
in the event logs can you determine from the "access denied" details, whether it is access denied dude to account lockout or because the account being used is not listed in the ACL list for the services in question?

For the afflicted services, you could create a domain admin account, for example, with no password expiry and use that as the account associated with those services.  This is only a troubleshooting trick not a fullon solution.

Looking forward to your reply.
0
 

Author Comment

by:scubadiver_dave
Comment Utility
The event logs only show x service failed to start due to access denied. All of these services require the NETWORK SERVICE and LOCAL SERVICE accounts to be in the security section but they aren't there. They keep being removed. I've tried in past troubleshooting with MS to give these services EVERYONE full controll and that didn't help either.
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
Remove any AV installed this seems to be an AV or virus issue. I would recommend demoting one of the DCs doing a reformat of the server then re-promote server to see if this is an issue with the actuall install
0
 

Author Comment

by:scubadiver_dave
Comment Utility
AV isn't installed on these servers. Well, it was on two of them but I removed it and didn't install it on the 3rd box. This issue has been reproduced on 3 different servers all built the same way and didn't occur until after Windows Update AFTER DCPROMO.
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
Again I would demote one server do a clean install of the server to see if this makes a difference if it does then you know there was a configuration or install issue. Could have been Windows Updates issue that overall caused the problem but a reinstall is the only way to know if the issue was this type of issue
0
 

Author Comment

by:scubadiver_dave
Comment Utility
Ok. I'm demoting DC3 now to a member server. I'll run updates after and see how it goes. Do you think I should promot it again today or let it sit for a day?
0
 

Author Comment

by:scubadiver_dave
Comment Utility
So, upon demotion and reboot DC3 is now a member server. I log in and all these services are stopped and can't start due to Access Denied. The NETWORK SERVICE and LOCAL SERVICE accounts are not in the security for the needed services. UGH!
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
What I was saying was reinstall the OS itself.
0
 
LVL 6

Expert Comment

by:SHIELD1
Comment Utility
re-nstalling the OS would resolve the problem on that server but the problem is highlighted either by installing a particular windows update or by joining the server to active directory.  Regular deletion of the required accounts shows that the culprit has an overriding default setting I.E. group or domain policy.

In my opinion, the application of an update that is then made obsolete by running a script is unlikely to then persist and make those accounts disappear again.

If you did re-install the OS, before any updates are run, confirm that the services and accounts are fine, then run DCPROMO - if the account are lost after joining AD then group policy or similar will be the cause.

Hopefully I have explained myself properly!
0
 

Author Comment

by:scubadiver_dave
Comment Utility
Shield1:

I'm going to resinstall the OS with 2008 R2. It's a completely different ISO image so I can remove that from the possible problems. I'll report back on the services after it's built and promoted.

We do have a 2003 DC in place still that's been here for years and it's never had this issue. Only 2008.
0
 

Author Comment

by:scubadiver_dave
Comment Utility
Ok, I have a Server 2008 R2 server built. When I run DCPROMO it's telling me I have to run the forestprep and domainprep. I've already done this but in 2008 standard only. Does 2998 R2 require a different domain/forest prep?
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 

Author Comment

by:scubadiver_dave
Comment Utility
Got it. Thanks.

Update...

Again, after I run DCPROMO the server is fine after the first reboot. Then I run Windows Update which installed 3 updates (KB981550, KB982526, KB2398632) After the reboot the system time was hours off and all the same services fail to start. I reset the time to current time, reboot and the time changes back and the services still don't start.

I can only see 2 of the 3 updates that just got installed in order to uninstall them. I was NOT able to find/uninstall KB982526.
So, this is a NEW install from a different ISO image and even a different version of Windows 2008. How can I find where KB982526 is installed so I can see about removing that?
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
You should be able to view the update.
0
 

Author Comment

by:scubadiver_dave
Comment Utility
This must be a policy issue. I'm looking over the Default Domain Controller Policy and under Security Settings and the Registery and File System keys all the settings have

CREATOR OWNER
AUTHENTICATED USERS
SYSTEM
ADMINS

but none of them have NETWORK SERVICE or LOCAL SERVICE ACCOUNTS

I don't want to mess with these until I get some feedback though.

I'm not sure why this problem isn't happening on my 2003 DC though.
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
I saw a link the other other about this type of issue with GPO removing the Network Service and Local Service accounts which was causing an issue on Windows 2008 Server but not on Windows 2003 server it was on Technet.
0
 

Author Comment

by:scubadiver_dave
Comment Utility
Do you have that link?
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
I am trying to find it.
0
 
LVL 6

Expert Comment

by:SHIELD1
Comment Utility
the distribution of those missing services that you are noticing to be absent *offhand* has been mentioned to be a security related omission from 2008 as the default setting as those accounts in the "wrong" place can allow attackers to take control of your system remotely.

In many instances such as with IIS permissions it is necessary to add those network and service accounts in manaually.  However, as I mentioned earlier, if they are manually re-instated using that script you referred to then it is likely that the default policy is indeed removing them.

Attacking the policy, providing you find it somewhere under allowed to log on locally or such, would not be detrimental.  However, Microsoft have provided cunning "help me" links to each of the policy settings that will tell you (when clicked) what the best practice policy is for enabling, disabling, or adding, to said policy.

In any event, look to see whether those affected accounts and services are listed in the security settings for the default policy, group policy, and default domain controller policies - it is my suspicions one or more of them will not be in the ACL list for "allowed" groups somewhere.

Let us know how you get on one way or the other :)
0
 

Author Comment

by:scubadiver_dave
Comment Utility
I checked the Default Domain Controller Policy and the NETWORK SERVICE and LOCAL SERVICE accounts are NOT anywhere it seems. If this is by design for 2008 I don't understand why these services require those accounts to start.

What am I supposed to do here? It looks like member servers use the NETWORK SERVICE account to start the BFE service but domain controllers aren't supposed to?
0
 
LVL 59

Expert Comment

by:Darius Ghassem
Comment Utility
If we think it is GPO then we can reset the GPO.
0
 

Author Comment

by:scubadiver_dave
Comment Utility
What do you mean reset the GPO? I think it's specifically the default domain controller policy that is the issue and it only affects 2008.
0
 
LVL 6

Expert Comment

by:SHIELD1
Comment Utility
Sorry to leave this so long my friend - when a server is promoted to AD it is taken for granted that it will be dealing with user/domain accounts etc and therefore the distribution and availability of certain systems (and how the file security is propagated) is all changed to lock the server down.  (this is visible when you watch AD installing, I know I'm a sad IT geek)

It is under this assumption that I put forward that these normally available services are omitted once the servers join AD.  If this is the case, you can verify for or against this suggestion by running the 2008 best practices security analyser .  As you walk through it, you will be asked for the roles your server will host.

During your interrogation you will see the pros and cons for each service and why they are suggested for removal.  Take for example, if you say yes to web server it will sort your firewall rules but disable file and printer sharing.  This is true of many network services and roles - at the same time you can enable other services and make educated guesses as to why certain services are not available in the domain policy allowed services and accounts.

I'm not MCSE I learned Win2K8 by telling the analyser it was wrecking my server build but then reading up on why Microsoft had implemented these new changes.  Until an MCSE or guru comes up with a direct answer you have nothing to lose!

(BE WARNED - the security adviser implements changes that may see you re-installing windows to correct/undo!  It's been a while I CANNOT say for certain if there was an undo option, therefore do not apply the changes just read why these changes are being suggested)

You mentioned that the services were not in the policy when you looked, I also said to you that each policy has a "read more" link that explains the pros and cons of changing these policy settings.  In addition, for each policy you are uncertain about - google it, many experts will have commented on what you need enabled or disabled etc to make your system work.

You can choose to add the services or service account in where they are missing for example (off the top of my head) "allowed to run as a service" or "allowed log on locally" and then add the missing accounts.

When I suggested creating a domain admin account and then using this account to run the services you said you tried this with the admin account to no avail it stands to reason that the admin account had not been granted the rights in domain controller policy.

This is a long speculative answer but if you're stuck we can run a remote session to your server with your consent using RDP and an open port 3389 on your router/firewall and I assume you have a static live IP?  You can monitor the session if you have any concerns.

Thanks
0
 

Accepted Solution

by:
scubadiver_dave earned 0 total points
Comment Utility
Update.

I called MS about this because it's been going on for so long. The Default Domain Controller policy left over from 2003 is pushing policy settings specifically for DCs and when these policies are enforced on a 2008 DC the accounts are removed. MS wasn't sure why these registry policies exist on the 2003 DC as they don't on theirs or the 2008 DC's but for some reason there are a whole bunch of registry policies there that are there but missing the accounts needed for the 2008 DC services.

We blocked the Default Domain Controller policy and added the necessary accounts for various services in order to get them  to start.I'll paste in what we did for anyone else with this problem down the road. Apparently, we don't need the Default Domain Controller Policy unless we're actually using it for something, which we aren't. I just don't know why these entries are present in this policy if they aren't supposed to be and I never set them. I'll attach a screen shot of the policy as well that was the cause of the problem.

Thanks for all the help though.

Dave
From Microsoft:
NOTE: These services are local machine accounts and not domain accounts! So change the source from Forest to local server

• Regarding the DPS service, we need to add “NT Service\BFE” account the following allow permissions on HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BFE:
Query Value
Set Value
Create Subkey
Enumerate Subkeys
Notify
Read Control

• Regarding the DPS service, we need to add “NT Service\DPS” account the following allow permissions on HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DPS:
Query Value
Set Value
Create Subkey
Enumerate Subkeys
Notify
Read Control

Also it was necessary to give the same permissions to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\WDI\Config

• Regarding the Windows Firewall service, we need to add “NT Service\mpssvc” account the following allow permissions on HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\mpssvc:
Query Value
Set Value
Create Subkey
Enumerate Subkeys
Notify
Read Control

Also it was necessary to give the same permissions to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\SharedAccess

NT Service\DPS” account the following allow permissions on HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DPS:
Query Value
Set Value
Create Subkey
Enumerate Subkeys
Notify
Read Control

Also it was necessary to give the same permissions to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\WDI\Config


dcpolicy.jpg
0
 

Expert Comment

by:TCNinja
Comment Utility
I seem to be having the same issue but I'm a bit confused about the resolution.   Is there some sort of article anywhere that provides proper procedures on fixing this.   I'm having this issue on two Windows Server 2008 R2 DCs that joined an existing 2003 Domain.  ForestPrep and DomainPrep were executed successfully.

Any help would be appreciated.
0
 
LVL 6

Expert Comment

by:SHIELD1
Comment Utility
He basically said that the 2003 DC was forcing GPO on the 2008 servers.  The solution was to stop the use of DOMAIN GPO and / or reset it.  I'd say stop GPO on the 2003 box and then reset it on 2008 servers.

The solution above then goes on to say how to add/give the missing service accounts to the affected services.

Is there a link that explains it?  Dunno - the fix came from Microsoft direct the guy said...
0
 

Expert Comment

by:TCNinja
Comment Utility
So,  I should just remove the Default Domain Controller Policy from the Domain Controller Org Unit?
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Sometimes drives fill up and we don't know why.  If you don't understand the best way to use the tools available, you may end up being stumped as to why your drive says it's not full when you have no space left!  Here's how you can find out...
OfficeMate Freezes on login or does not load after login credentials are input.
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

6 Experts available now in Live!

Get 1:1 Help Now