Link to home
Start Free TrialLog in
Avatar of graye
grayeFlag for United States of America

asked on

Can't get BuildExplicitAccessWithName API to work in 64-bit VB.Net

I've tried every conceivable P/Invoke signatures to both the API and 2 strutures and I still can't get BuildExplicitAccessWithName API to work when compiled against the "AnyCPU" or "x64" target.

Help!

BTW: The code below works just fine when compiled against the "x86" target
Imports System.Runtime.InteropServices
 
Module Module1
 
    Private Const CONTAINER_INHERIT_ACE As Integer = &H2
    Private Const OBJECT_INHERIT_ACE As Integer = &H1
    Private Const SET_ACCESS As Integer = &H2
 
    Private Const SYNCHRONIZE As Integer = &H100000
    Private Const READ_CONTROL As Integer = &H20000
    Private Const WRITE_DAC As Integer = &H40000
    Private Const WRITE_OWNER As Integer = &H80000
    Private Const STANDARD_RIGHTS_READ As Integer = (READ_CONTROL)
    Private Const STANDARD_RIGHTS_WRITE As Integer = (READ_CONTROL)
    Private Const DELETE As Integer = &H10000
    Private Const FILE_DELETE_CHILD As Integer = &H40
    Private Const FILE_ALL_ACCESS As Integer = &HF0000 Or SYNCHRONIZE Or &H1FF
 
    'typedef struct _TRUSTEE {
    '  PTRUSTEE pMultipleTrustee;
    '  MULTIPLE_TRUSTEE_OPERATION MultipleTrusteeOperation;
    '  TRUSTEE_FORM TrusteeForm;
    '  TRUSTEE_TYPE TrusteeType;
    '  LPTSTR ptstrName;
    '} TRUSTEE, *PTRUSTEE;
    <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto)> _
    Public Structure TRUSTEE
        Dim pMultipleTrustee As IntPtr
        Dim MultipleTrusteeOperation As Integer
        Dim TrusteeForm As Integer
        Dim TrusteeType As Integer
        Dim ptstrName As String
    End Structure
 
    'typedef struct _EXPLICIT_ACCESS {
    '  DWORD grfAccessPermissions;
    '  ACCESS_MODE grfAccessMode;
    '  DWORD grfInheritance;
    '  TRUSTEE Trustee;
    '} EXPLICIT_ACCESS, *PEXPLICIT_ACCESS;
    <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto)> _
    Public Structure EXPLICIT_ACCESS
        Dim grfAccessPermissions As Integer
        Dim grfAccessMode As Integer
        Dim grfInheritance As Integer
        Dim Trustee As TRUSTEE
    End Structure
 
    'VOID BuildExplicitAccessWithName(
    '  PEXPLICIT_ACCESS pExplicitAccess,
    '  LPTSTR pTrusteeName,
    '  DWORD AccessPermissions,
    '  ACCESS_MODE AccessMode,
    '  DWORD Inheritance
    ');
    Declare Auto Sub BuildExplicitAccessWithName Lib "AdvAPI32.DLL" ( _
        ByRef pExplicitAccess As EXPLICIT_ACCESS, _
        ByVal pTrusteeName As String, _
        ByVal AccessPermissions As Integer, _
        ByVal AccessMode As Integer, _
        ByVal Inheritance As Integer _
    )
 
    Sub Main()
        Dim ea As New EXPLICIT_ACCESS
 
        Console.WriteLine("starting")
        BuildExplicitAccessWithName(ea, "Users", FILE_ALL_ACCESS, SET_ACCESS, CONTAINER_INHERIT_ACE Or OBJECT_INHERIT_ACE)
        Console.WriteLine("finished")
        Console.WriteLine("ea.Trustee.ptstrName='" & ea.Trustee.ptstrName & "'")
 
        Console.ReadLine()
    End Sub
 
End Module

Open in new window

Avatar of Corey Scheich
Corey Scheich
Flag of United States of America image

It appears that the API you are calling connects to an x32 process.  Is there any need for it to run as a 64 bit process?  A 32 bit process will run the same on a 64 and a 32 bit machine.   There are many reasons you may want it as a 64 bit process (RAM usage, Making API calls to another 64 bit process).


If you actually need to run as a x64 process it requires remoting to call 32bit API.  I would suggest an abstraction class that makes all the 32 bit calls on the 32 bit side of the fence and passes simple data types to the 64bit process.  I found this blog that lays out how to use IPC remoting the example is in c# but could be easily translated to vb.net using one of the many free translators out there.

http://weblogs.asp.net/israelio/archive/2005/01/04/346180.aspx

a translator

http://www.developerfusion.com/tools/convert/csharp-to-vb/
Avatar of kekal
kekal

I do not see how the code is connecting to an x32 process.  The call to BuildExplicitAccessWithName should work the same from either an x64 or x86 application.  Like graye, I have tried everything I know to get the call to work as an x64 bit application.

Having to make some progress, I decided to go with a x86 build.  However, I ran in to other APIs that I need to use that are unique to x64.
I was doing some more research--I hate it when I get something to work--and found a note on pinvoke.net on the EXPLICT_ACCESS structure that said Pack:=4 does not work on an x64 bit system.  

See http://www.pinvoke.net/default.aspx/Structures/EXPLICIT_ACCESS.html

I changed the Pack:=4 to Pack:=0 on both the EXPLICIT_ACCESS and TRUSTEE structures and the application doesn't crash any more.

The code does not appear to work (I haven't researched why yet), but we may be on to something.
      <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto, Pack:=0)> _
      Private Structure EXPLICIT_ACCESS
         Private grfAccessPermissions As Integer
         Private grfAccessMode As Integer
         Private grfInheritance As Integer
         <MarshalAs(UnmanagedType.Struct)> Private Trustee As TRUSTEE
      End Structure
      <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto, Pack:=0)> _
      Private Structure TRUSTEE
         Private pMultipleTrustee As Integer ' must be null, so no need for IntPtr
         Private MultipleTrusteeOperation As Integer
         Private TrusteeForm As Integer
         Private TrusteeType As Integer
         Private ptstrName As String
      End Structure

Open in new window

Avatar of graye

ASKER

Yeah, I'm the one who intially put the API signatures on the P/Invoke wiki  (with the incorrect Pack:=4 and the pMultipleTrustee as Integer)
My example here has the corrected signatures.   I'll try the <MarshalAs(UnmanagedType.Struct)>  attribute and see what I can get out of it
From what I can tell, the 64-bit version of the API doesn't terminate the ptstrName string value with a null.   So, I think I'm gonna submit this as a bug to Microsoft.
I can get it to do something.  Taking what you just wrote (ptstrName not being null terminated) with what I've seen, I think a bug report is justified.

When I run my test code that denies terminate to the current user for a process (graye helped me figure that out in a previous Experts Exchange question), as x86, it works.  When I run the code as x64 (outside of the Visual Studio environment, compiled as either debug or release), the username parameter of the API call is ignored and "BUILTIN" is used.  While this works for my purposes (for version 1 of my app), I will get to get this figured out sooner or later.

If I run the x64 bit code from within Visual Studio, the call works (no win32 error) but the the EA does not appear to be valid.  
Okay...this is getting stranger.

If my test application starts in Sub Main, the call fails.  If the startup object is Form1, the call works.  I have duplicated this on two different computers (both running Vista x64).

I do not know what I did, but I can no longer duplicate my previous post.

Wierd.
Me again....

I'm also getting different results between Release and Debug modes.
What about using Run as Admin on vista 64?
what is the trust level of the project?
Avatar of graye

ASKER

Nope, UAC is off
I was wrong about it acting differently between debug and release modes.  I did a step-by-step compare between and found that debug had optimizations turned off (the default) while release had it turn on (again, the default).

If I turn optimizations off, it does not crash.

So, I am where we started.  
* On an x86 system (XP and Windows 7) in debug or release with optimizations off, it appears to work.
* On a x64 system  (Vista) in debug or release with optimizations off, it does not work--best I can tell it is not returning an DACL or Owner (I'm haven't played with groups or SACLs)

BTW, I am compiling for "AnyCPU".  I may have been seeing some different results if I compiled for x86 or x64 and tested on the appropriate OSs.  

Unless there is a significant bug as it relates to x64 bit systems, I believe that something is wrong with the API signatures or structure layout.  Like graye, I've tried darn near everything combination with no real luck.

I've included the code that works (at least for me) below.  It is not very different from grayes.  
      <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto, Pack:=0)> _
      Private Structure EXPLICIT_ACCESS
         Public grfAccessPermissions As Integer
         Public grfAccessMode As Integer
         Public grfInheritance As Integer
         <MarshalAs(UnmanagedType.Struct)> Public Trustee As TRUSTEE
      End Structure
      <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Auto, Pack:=0)> _
      Private Structure TRUSTEE
         Public pMultipleTrustee As Integer ' must be null, so no need for IntPtr
         Public MultipleTrusteeOperation As Integer
         Public TrusteeForm As Integer
         Public TrusteeType As Integer
         Public ptstrName As String
      End Structure
 
 
      <Security.Permissions.SecurityPermission(Security.Permissions.SecurityAction.Assert)> _
      Private Declare Auto Sub BuildExplicitAccessWithName Lib "AdvAPI32.DLL" ( _
         ByRef pExplicitAccess As EXPLICIT_ACCESS, _
         ByVal pTrusteeName As String, _
         ByVal AccessPermissions As Integer, _
         ByVal AccessMode As Short, _
         ByVal Inheritance As Integer)
 
 
Sub Main()
 
         Dim myEA As New EXPLICIT_ACCESS
 
         ' Build an explicit access structure
         BuildExplicitAccessWithName(myEA, UserName, iPerm, AccessMode.DENY_ACCESS, 0)   'CONTAINER_INHERIT_ACE Or OBJECT_INHERIT_ACE
 
End sub

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of graye
graye
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial