CreateMutex problem

I have a COM dll that is run inprocess by two separate executables - one of the executables is run as a service.  The OS that both executables are running on is NT4.  That said, now here's the problem:

In the dll, there is an interface that has a call to CreateMutex which succeeds if the service executable is not running and the other executable is just accessing the dll standalone.

However if the service executable is started, the call to CreateMutex fails if the non service executable tries to call on the dlls interface.  This stops the non service excutable program from working properly.

Can anyone give me the solution to this problem, so that both executables can access the dlls methods at the same time?


Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

You'll have to provide tha appropriate security settings in order for this to work:

HANDLE                      hMutex;

PSID                        psidWorldSid    =   NULL;


    psidWorldSid    =   ( PSID) LocalAlloc  (   LPTR,
                                                GetSidLengthRequired    (   1)

    InitializeSid   (   psidWorldSid,   &siaWorldSidAuthority,  1);

    *(  GetSidSubAuthority  (   g_psidWorldSid, 0)) =   SECURITY_WORLD_RID;

    InitializeSecurityDescriptor    (   &sd,    SECURITY_DESCRIPTOR_REVISION);

    SetSecurityDescriptorGroup      (   &sd,    g_psidWorldSid, TRUE);

    ZeroMemory  (   &sa,    sizeof  (   SECURITY_ATTRIBUTES));

    sa.nLength              =   sizeof  (   SECURITY_ATTRIBUTES);
    sa.lpSecurityDescriptor =   &sd;
    sa.bInheritHandle       =   FALSE;

    //  set this SD on the mutex..
    hMutex  =   CreateMutex (   &sa,

Feel free to ask if you need more information!

(BTW: This snippet is taken from 'production' code :o)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
carlpaddickAuthor Commented:
jkr thanks for your reply and your code.  Hmm, think I might need some help with this one!  I have no idea how NT4 security works behind the scenes.

Basically I want to grant access to everyone so that any executable can use the dll.  So do I have to assign a null DACL in the security setting?  If so, how do I do this?  If not, could you help me grant this access please?  Thanks
>>Basically I want to grant access to everyone so that any
>>executable can use the dll.  So do I have to
>>assign a null DACL in the security setting?

A NULL DACL would also be an option, but a 'WORLD' grop SID serves the same purpose.

>>I have no idea how NT4 security works behind the scenes.

I recommend ('Windows NT Security in Theory and Practice') and the articles linked from this one as a lecture :o)
Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

carlpaddickAuthor Commented:
I've just added your code from above jkr, but the problem in the dll still remains.  My hMutex HANDLE value is still null after the CreateMutex call is made.  I really don't understand what is happening here.  I'm using the same login account/password and session to stop and start the service, and run the stand alone executable.  All the service .exe does at startup is instantiate an instance of the dll for later use, but it doesn't call any of it's methods.  Running the standalone .exe from either the command line or within Developer Studio produces the same problem in the DLL.  Any ideas?
>>My hMutex HANDLE
>>value is still null after the CreateMutex call is made

Aaah, I think I got the idea what might be wrong - there's another reason why that call might fail:

    hMutex  =   CreateMutex (   &sa,

    if  (   INVALID_HANDLE_VALUE    ==  hMutex)
            if  (   ERROR_ALREADY_EXISTS    ==  GetLastError    ())
                    hMutex  =   OpenMutex   (   MUTEX_ALL_ACCESS,

                    if  (   INVALID_HANDLE_VALUE    ==  g_hDbgFileLock)
                            //  error
carlpaddickAuthor Commented:
Just added your testing code above jkr.  GetLastError() returns 5 - "access denied" so it doesn't drop into the OpenMutex test.

I don't know if this would have any effect or not but in the Services utility, the executable file running as a service is started using the following startup parameters:

Log on as:

System Account (selected)
Allow service to interact with desktop (checked)
carlpaddickAuthor Commented:
jkr, just leaving work for the day...please keep me posted with any news and I'll speak to you tomorrow.
carlpaddickAuthor Commented:
Hi jkr,

Just going back to your most recent comment posted, I think you might have been thinking that the executable running as a service, might have called the Interface method within the dll that makes the call to CreateMutex when it initially starts up.

This however isn't the case.  In fact it never makes a call on this interface in its entire running lifetime.  It's only the separate excutable (which is run from the command line) that makes this call.

Hope this may help.
>>I think you might have been thinking that the executable
>>running as a service, might have called the Interface
>>method within the dll that makes the call to CreateMutex
>>when it initially starts up.

Yes, that's what I actually thought - the code shown above is used by a DLL that is called by either a service or a user mode app, and the mutex' purpose actually is to guard access to a log file...
carlpaddickAuthor Commented:
So I wonder why I'm still getting error code 5 on the CreateMutex call?  The same user (me logged in as a user who has administrator rights) starts the service .exe and runs the command line .exe.

Can you think of anything else that the service or Windows NT might be doing to this dll file to stop me using it?
>>Can you think of anything else that the service or
>>Windows NT might be doing to this dll file to stop
>>me using it?

Actually, no. This code should work fine, regardless whether it's called within a DLL or an executable...
carlpaddickAuthor Commented:
jkr - hi,

I'm going to accept your original proposed answer, as I'm sure it works in most cases.  However for mine this hasn't been the case, and I still have been getting the error code 5 on the CreateMutex call.

My only workaround has been to take out the interface method and put it into its own normal Win32 dll.  This of course now works fine, and cures the problem.

Thanks anyway for you help,

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
System Programming

From novice to tech pro — start learning today.