Link to home
Start Free TrialLog in
Avatar of scubadiver_dave
scubadiver_daveFlag for United States of America

asked on

Many services fail to start on new 2008 DC- Access Denied

Hi,

I built 3 new 2008 standard domain controllers from scratch. All are virtual machines on VMWare. On each server after adding the DC role, dcpromo, etc. and then running Windows updates a bunch of services fail to start due to access denied.

Failing services:
Base Filtering Engine (BFE)
IPSec Policy Agent
Windows Time
IKE and AuthIP IPSec Keying Modules
DHCP Client
Distributed Transaction Coordinator (DTC)
Diagnostic Policy
Network List Service
Network Location Awareness
Windows Firewall

I had an email ticket with Microsoft on this and wasted 2 weeks of time. They kept focusing on the permissions on the BFE service. The neccessary permissions that were NOT in place were for the NETWORK SERVICE and LOCAL SERVICE. I eventually found a link to an article on this very problem. http://techruckus.com/forum/vista-dhcp-base-filtering-service-access-denied-t-t91.html

I followed the instructions and it fixed the problem. Basically, you install the SubInACL and run a script which recreates all the permissions and then the services start.

Now, the problem....this problem has reappeared on each of these 3 DCs. I've had to rerun this script now a second time on each DC. I don't think it is a group policy problem because after a reboot all the services still are ok. It's at some point a few days later that the services fail or stop and can't start because these NETWORK SERVICE LOCAL SERVICE accounts don't have access. The accounts just aren't even there. So, I have to rerun the script and that fixes it.

I'm very concerned about this as when these services are stopped it causes problems in DC replication and Exchange!

Any thoughts on why these permissions keep being removed?
Avatar of willettmeister
willettmeister

Do you have any other services on you network that woudl monitor accounts and automatically change/update them.  I know some security programs do that.  

Avatar of scubadiver_dave

ASKER

Not that I can think of. I don't even have AV on these boxes yet.
Avatar of Firmin Frederick
in the event logs can you determine from the "access denied" details, whether it is access denied dude to account lockout or because the account being used is not listed in the ACL list for the services in question?

For the afflicted services, you could create a domain admin account, for example, with no password expiry and use that as the account associated with those services.  This is only a troubleshooting trick not a fullon solution.

Looking forward to your reply.
The event logs only show x service failed to start due to access denied. All of these services require the NETWORK SERVICE and LOCAL SERVICE accounts to be in the security section but they aren't there. They keep being removed. I've tried in past troubleshooting with MS to give these services EVERYONE full controll and that didn't help either.
Remove any AV installed this seems to be an AV or virus issue. I would recommend demoting one of the DCs doing a reformat of the server then re-promote server to see if this is an issue with the actuall install
AV isn't installed on these servers. Well, it was on two of them but I removed it and didn't install it on the 3rd box. This issue has been reproduced on 3 different servers all built the same way and didn't occur until after Windows Update AFTER DCPROMO.
Again I would demote one server do a clean install of the server to see if this makes a difference if it does then you know there was a configuration or install issue. Could have been Windows Updates issue that overall caused the problem but a reinstall is the only way to know if the issue was this type of issue
Ok. I'm demoting DC3 now to a member server. I'll run updates after and see how it goes. Do you think I should promot it again today or let it sit for a day?
So, upon demotion and reboot DC3 is now a member server. I log in and all these services are stopped and can't start due to Access Denied. The NETWORK SERVICE and LOCAL SERVICE accounts are not in the security for the needed services. UGH!
What I was saying was reinstall the OS itself.
re-nstalling the OS would resolve the problem on that server but the problem is highlighted either by installing a particular windows update or by joining the server to active directory.  Regular deletion of the required accounts shows that the culprit has an overriding default setting I.E. group or domain policy.

In my opinion, the application of an update that is then made obsolete by running a script is unlikely to then persist and make those accounts disappear again.

If you did re-install the OS, before any updates are run, confirm that the services and accounts are fine, then run DCPROMO - if the account are lost after joining AD then group policy or similar will be the cause.

Hopefully I have explained myself properly!
Shield1:

I'm going to resinstall the OS with 2008 R2. It's a completely different ISO image so I can remove that from the possible problems. I'll report back on the services after it's built and promoted.

We do have a 2003 DC in place still that's been here for years and it's never had this issue. Only 2008.
Ok, I have a Server 2008 R2 server built. When I run DCPROMO it's telling me I have to run the forestprep and domainprep. I've already done this but in 2008 standard only. Does 2998 R2 require a different domain/forest prep?
Got it. Thanks.

Update...

Again, after I run DCPROMO the server is fine after the first reboot. Then I run Windows Update which installed 3 updates (KB981550, KB982526, KB2398632) After the reboot the system time was hours off and all the same services fail to start. I reset the time to current time, reboot and the time changes back and the services still don't start.

I can only see 2 of the 3 updates that just got installed in order to uninstall them. I was NOT able to find/uninstall KB982526.
So, this is a NEW install from a different ISO image and even a different version of Windows 2008. How can I find where KB982526 is installed so I can see about removing that?
You should be able to view the update.
This must be a policy issue. I'm looking over the Default Domain Controller Policy and under Security Settings and the Registery and File System keys all the settings have

CREATOR OWNER
AUTHENTICATED USERS
SYSTEM
ADMINS

but none of them have NETWORK SERVICE or LOCAL SERVICE ACCOUNTS

I don't want to mess with these until I get some feedback though.

I'm not sure why this problem isn't happening on my 2003 DC though.
I saw a link the other other about this type of issue with GPO removing the Network Service and Local Service accounts which was causing an issue on Windows 2008 Server but not on Windows 2003 server it was on Technet.
Do you have that link?
I am trying to find it.
the distribution of those missing services that you are noticing to be absent *offhand* has been mentioned to be a security related omission from 2008 as the default setting as those accounts in the "wrong" place can allow attackers to take control of your system remotely.

In many instances such as with IIS permissions it is necessary to add those network and service accounts in manaually.  However, as I mentioned earlier, if they are manually re-instated using that script you referred to then it is likely that the default policy is indeed removing them.

Attacking the policy, providing you find it somewhere under allowed to log on locally or such, would not be detrimental.  However, Microsoft have provided cunning "help me" links to each of the policy settings that will tell you (when clicked) what the best practice policy is for enabling, disabling, or adding, to said policy.

In any event, look to see whether those affected accounts and services are listed in the security settings for the default policy, group policy, and default domain controller policies - it is my suspicions one or more of them will not be in the ACL list for "allowed" groups somewhere.

Let us know how you get on one way or the other :)
I checked the Default Domain Controller Policy and the NETWORK SERVICE and LOCAL SERVICE accounts are NOT anywhere it seems. If this is by design for 2008 I don't understand why these services require those accounts to start.

What am I supposed to do here? It looks like member servers use the NETWORK SERVICE account to start the BFE service but domain controllers aren't supposed to?
If we think it is GPO then we can reset the GPO.
What do you mean reset the GPO? I think it's specifically the default domain controller policy that is the issue and it only affects 2008.
Sorry to leave this so long my friend - when a server is promoted to AD it is taken for granted that it will be dealing with user/domain accounts etc and therefore the distribution and availability of certain systems (and how the file security is propagated) is all changed to lock the server down.  (this is visible when you watch AD installing, I know I'm a sad IT geek)

It is under this assumption that I put forward that these normally available services are omitted once the servers join AD.  If this is the case, you can verify for or against this suggestion by running the 2008 best practices security analyser .  As you walk through it, you will be asked for the roles your server will host.

During your interrogation you will see the pros and cons for each service and why they are suggested for removal.  Take for example, if you say yes to web server it will sort your firewall rules but disable file and printer sharing.  This is true of many network services and roles - at the same time you can enable other services and make educated guesses as to why certain services are not available in the domain policy allowed services and accounts.

I'm not MCSE I learned Win2K8 by telling the analyser it was wrecking my server build but then reading up on why Microsoft had implemented these new changes.  Until an MCSE or guru comes up with a direct answer you have nothing to lose!

(BE WARNED - the security adviser implements changes that may see you re-installing windows to correct/undo!  It's been a while I CANNOT say for certain if there was an undo option, therefore do not apply the changes just read why these changes are being suggested)

You mentioned that the services were not in the policy when you looked, I also said to you that each policy has a "read more" link that explains the pros and cons of changing these policy settings.  In addition, for each policy you are uncertain about - google it, many experts will have commented on what you need enabled or disabled etc to make your system work.

You can choose to add the services or service account in where they are missing for example (off the top of my head) "allowed to run as a service" or "allowed log on locally" and then add the missing accounts.

When I suggested creating a domain admin account and then using this account to run the services you said you tried this with the admin account to no avail it stands to reason that the admin account had not been granted the rights in domain controller policy.

This is a long speculative answer but if you're stuck we can run a remote session to your server with your consent using RDP and an open port 3389 on your router/firewall and I assume you have a static live IP?  You can monitor the session if you have any concerns.

Thanks
ASKER CERTIFIED SOLUTION
Avatar of scubadiver_dave
scubadiver_dave
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I seem to be having the same issue but I'm a bit confused about the resolution.   Is there some sort of article anywhere that provides proper procedures on fixing this.   I'm having this issue on two Windows Server 2008 R2 DCs that joined an existing 2003 Domain.  ForestPrep and DomainPrep were executed successfully.

Any help would be appreciated.
He basically said that the 2003 DC was forcing GPO on the 2008 servers.  The solution was to stop the use of DOMAIN GPO and / or reset it.  I'd say stop GPO on the 2003 box and then reset it on 2008 servers.

The solution above then goes on to say how to add/give the missing service accounts to the affected services.

Is there a link that explains it?  Dunno - the fix came from Microsoft direct the guy said...
So,  I should just remove the Default Domain Controller Policy from the Domain Controller Org Unit?