Start Free Trial

asked on

Hyper-V 2012 R2 - MPIO config error “Access Denied”

I am in the midst of migrating to a Hyper-V 2012 R2 cluster and am currently configuring the first host to connect to the iSCSI CSV via MPIO.

I "think" I have everything working properly but when I try to create a log file to verify via the MPIO properties (GUI) or invoke the "mpclaim -v" powershell command (or command prompt) while elevated as Admin, I get the following errors;

GUI - Failed to probe MPIO storage configuration. Access is denied.

Elevated Powershell/CMD - File creation failed. C:\Windows\System32\MPIO_Configuration.log. Error 5 Failed to write MPIO configuration to file. Access is denied.

I have only been able to locate one article relating to the same problem but the solution was not applicable to me. Someone made a reference to " Local Security Policy/Public Key Policies/Encrypting File System/Properties/Certificates" and to allow something there but when I go there on the local machine, there are no keys or anything.. just a message that says "No Encrypting File System Policies Defined".

Here's the article;

https://social.technet.microsoft.com/Forums/windowsserver/en-US/6526b8c8-0fa9-4b47-9c31-3463896ffd51/access-denied-trying-to-capture-mpio-config?forum=winserverfiles

Anyone have any insight into this?

Thank you in advance!

Did you actually enable the Allow EFS to generate self-signed certificates when a certification authority is not available setting in the Local Security Policy though?

- Right click Start button then click Run
- In the Run dialog box that appears type in secpol.msc then click OK
- Expand Public Key Policies on the left then right click on Encrypting File System
- Click on Properties

User generated image

- Select Allow in the General tab
- Click on the Certificates tab and confirm that Allow EFS to generate self-signed certificates when a certification authority is not available is ticked
- Configure the rest of the settings as desired, however the default settings should suffice.
- Reboot your server when done
- Try running the mpclaim command again

ASKER

Per my original posting, I already looked in Local Security Policy and within the Encryption File System key, but there is nothing there. See attached screenshot.

ASKER

If you look at my above post you'll note that I said you need to RIGHT CLICK on Encrypting File System, then click on Properties to see the settings.

User generated image

User generated image

User generated image

Please re-read my instructions in my previous post, I tried to be as detailed as possible.

ASKER

You are absolutely correct. My sincere apologies.

If you look at the timestamp, it was nearly midnight and so my attention to detail wasn't at 100% ;o)

I followed your instructions to the letter but I am still getting the same errors.

That's OK, it's happened to all of us! :)

Did you reboot your server after making the changes?

ASKER

Yep, I rebooted (per your instructions) but the problem persists.

I saw your article
I don't understand how EFS is relevant here in this scenario

U should enable MPIO support for ISCSi device from MPIO Properties and
after that your ISCSI device vendor ID should get added automatically to MPIO devices tab, if its not getting added, you should add it manually to MPIO Devices in MPIO Properties and reboot the server once to make it effective

Check below post for step by step
http://terrytlslau.tls1.cc/2013/09/configure-iscsi-connections-with-mpio_11.html

I hit this all the time in our cluster setups though we are using SAS based DAS and not iSCSI.

IIRC, cd \Temp on the C: drive.
mpclaim -v >Report.txt
Notepad Report.txt

It's vague and I'm not back into our system until the morning and can verify. There may be another -L switch for mpclaim for it to drop the log file in C:\Temp.

ASKER

@Mahesh, iSCSI devices were already added and MPIO configured (see attached screenshot).

@ Philip Elder, Sorry, I've already tried that. No matter where I output the txt file, the file will only contain the same "access denied" error.
Capture.PNG

ASKER

@Philip Elder - If I use the "-l" switch, the txt file is blank.

This is what I get:

User generated image

That is on a Hyper-V 2008 R2 cluster node.

ASKER

@Philip Elder - This is what I get when I do the same thing:

User generated image

Elevated CMD not PoSh.

ASKER

User generated image

What DSM is in operation here?

CMD --> MPIOCPL [Enter] --> Devices --> ?

Or, in Disk Management right click on any MPIO enabled drive and Properties then under the MPIO tab the DSM will display. Here you can click DETAILS. What MPIO version?

ASKER

User generated image

User generated image

That's what we have.

Are the nodes in their own OU structure or nested in with a bunch of other servers?

If possible, have an OU structure from the domain root for your cluster setup and configure the necessary firewall exceptions for cluster services and permissions.

I take it you are logged in as Local Admin or Domain Admin? Try logging in as the other and running in C:\Temp. Does that work?

ASKER

At the moment, I have just one Server 2012 R2 node attached to the iSCSI CSV. I do have a separate OU for the hyper-v nodes/csv nodes. All the 2008 R2 nodes and one 2012 R2 node are in there, plus the cluster object.

I have been logging in as a domain admin plus I tried another account with domain admin rights but to no success. I haven't tried a local admin account yet.

When you say necessary firewall rules, do you mean the ones that automatically get set when you install the Hyper-V role?

Is the Cluster Role installed?

ASKER

Yes, the Failover Cluster role is installed.

From what I know, the output file generated by the mpclaim command gets encrypted by EFS. I think in this case your Data Recovery Agent certificate has expired.

Can you please open the Group Policy Management Console on one of your Domain Controllers, go into the Default Domain Policy then go to this path?: Computer Configuration\Policies\Windows Settings\Security Settings\Public Key Policies\Encrypting File System

Has the certificate in there issued to Administrator expired? If so, you'll need to renew it. Follow the steps in this article which should still be applicable for Server 2008 and onwards: http://blogs.technet.com/b/askds/archive/2008/01/07/replacing-an-expired-dra-certificate.aspx

ASKER

It is expired (as of 2006).

It also looks like the default domain policy is disabled.

ASKER

The first option is greyed out and so I cannot export the private key per the instructions.

User generated image

The DEFAULT DOMAIN POLICY is disabled?!?! Huh?

Sounds like GP may be a rat's nest. :(

That policy should _never_ be disabled or edited. It contains a whole host of settings to keep a domain functional. I suggest comparing against a live DDP that has not been touched to verify that it will not further toast the domain.

It can be rebuilt if need be: KB556025: Manually Recreate Default GPOs.

ASKER

Yeah, I inherited this domain as is. The guy who built it and ran for 10 years is long gone.

I have a CA authority running on one of the DC's. Shouldn't EFS try to use that instead of self-signed?

I suggest addressing the Group Policy deficiencies before going anywhere with EFS.

ASKER

Yeah, the computer and User settings were set to disabled for whatever reason.

I switched them back on.

Is it safe to delete the expired DRA and create a new one? Unfortunately, I have never had to do this before.

It also looks like the default domain policy is disabled.

Never a good idea to disable this policy!

The first option is greyed out and so I cannot export the private key per the instructions.

This is because the private key for the DRA certificate can't be found.

User generated image

The original private key is stored in the Administrator profile of the first domain controller in the domain. If you don't have access to this Administrator profile then you may have issues decrypting files down the track.

Is it safe to delete the expired DRA and create a new one? Unfortunately, I have never had to do this before.

Yes it is, as long as you keep the exported certificate on file somewhere and have backup copies of it.

ASKER

I agree about the disabled Default Domain Policy, but as I said I inherited this "situation". I turned both user and computer settings back on yesterday. I've also exported the expired cert and created a new DRA. I've follow the instructions per the link above to the letter but the error remains.

I've created a separate OU for hyper-v nodes and CSV nodes. I've blocked inheritance on this OU and linked only specific GPO's that I want applied to these servers (which includes the default domain policy). Honestly, if MS didn't decide to start charging $500 per incident then I would've just opened a case with them ;) But I am not certain this issue is worth a $500 hit since it isn't technically stopping me from setting up my new cluster, it is simply preventing me from verifying my MPIO config.

I agree about the disabled Default Domain Policy, but as I said I inherited this "situation".

Understood, I wasn't blaming you just pointing it out for any others who may stumble across this thread.

When you re-enabled the Default Domain Policy did you run gpupdate /force on the Hyper-V host or reboot it?

ASKER

I sure did. Same error.

Since the latest 'request attention' notice went to all experts, you've reached the bottom of the barrel. No clue.

Recommend working with the experts in this question to get to your solution, as at first glance I see a whole lot of effort provided you.

Good luck.

Did you remove the changes you made to the local security policies which I outlined in my first comment after renewing the domain's Data Recovery Agent certificate?

ASKER

I switched the "File Encryption using EFS" option to NOT DEFINED.

After doing this, everything in the certificates tab is greyed out.

I ran gpupdate /force and tried mpclaim -v and got the same error.

ASKER

Doesn't look like this is going to be solved here.

I may need to just bite the bullet and open a case with MS. Maybe I'll get lucky and it'll turn out to be something that warrants a "freebie".

Did you run Command Prompt as an Administrator then changed directory to some other folder you create on the Hyper-V host's C: drive?

- Create a new folder in the root of the C: drive and call it Test
- Open Command Prompt as an Administrator
- Type in cd c:\temp
- Try the mpclaim -v command

If that doesn't work then I'm out of ideas unfortunately. I tested with an environment that had a similar issue recently which had 2008 R2 cluster nodes and got around it by renewing the DRA certificate.

ASKER

Yes sir.

I created a folder called temp at C: root and ran those commands via cmd and powershell with elevated permissions.

No dice.

I appreciate all the effort. I am going to talk to my teammate and see if I can convince him to open a case with MS on this. The issue doesn't "appear" to be blocking or interfering with my migration to a 2012 R2 cluster "so far".

Again, I appreciate everyone's help!

Alright let us know how you go with MS if you do open a case as I'm out of ideas unfortunately.

As you said this doesn't actually stop your cluster from working, you just can't verify your MPIO configuration.

ASKER CERTIFIED SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER

It turned out the error I was getting was not the fault of the OS at all. It was due to the iSCSI storage being faulty.

Our storage connection of choice is Direct Attached SAS. Each cable connection is 24Gb of essentially zero latency bandwidth. Each node has a minimum of two for 48Gb of aggregate bandwidth.

Our two node Scale-Out File Server clusters are 96Gb of aggregate bandwidth while our four nodes are 192Gb of aggregate bandwidth.

No FC or iSCSI complications. It's straightforward simple to set up and deploy.