Link to home
Start Free TrialLog in
Avatar of RichieHindle
RichieHindle

asked on

DLL/COM redirection with .exe.local only half-working

We have a windows application, x.exe, which uses a third-party OCX, y.ocx.  We install x.exe and y.ocx in the same directory and we register y.ocx (because it can be used by other components outside of x.exe's directory).  I'd like to use an x.exe.local file to ensure that the local version of y.ocx is always used by x.exe, even when multiple versions of the application are installed.

If I install two versions of the applications into directories One and Two, with no .local files, I can verify with File Monitor that running One/x.exe after installing Two will pick up Two/y.ocx.  As I understand it, creating One/x.exe.local ought to make One/x.exe load One/y.ocx, but it doesn't.  Using File Monitor I can see that running One/x.exe now checks that One/y.ocx exists (which wasn't happening before I created the .local file) but then loads Two/y.ocx.  x.exe is an MFC application developed with VC6.  Where am I going wrong?

(I need a solution for Win2000 as well as XP.  I can use a .manifest on XP and it all works.  The problems I'm having with .local happen on both XP and on 2000, and I need a solution for 2000.)

(Registering y.ocx by writing just the filename rather than the full pathname into the registry makes it work, but I have to put the full pathname in because y.ocx can be used by other components outside of x.exe's directory.)
Avatar of jkr
jkr
Flag of Germany image

I am surprised that it is even 'half' working. The complete path of the DLL is written to the registry upon registration, so '.local' has no effect on that if that path does not match that one anyway.
Avatar of RichieHindle
RichieHindle

ASKER

As I understand it, the presense of a .local file should cause the COM loader to first try ignoring the directory piece of the pathname in the registry, and loading the library from the executable's own directory.  Only if the library isn't there will it use the full pathname.  (File Monitor confirms that the .local file is having an effect something like this, but not for the whole load operation.)
>>As I understand it, the presense of a .local file should cause the COM loader to first try ignoring the directory piece

No. There is no such thing as a COM loader, the DLL is loaded via 'LoadLibrary()' from the location specified in the registry. So if that is

c:\Two\y.ocx

for what reason should

c:\One\x.exe.local

affect that? Or, in other words, since there is no way to find out if any other apps reference a similar module, the .local file has absolutely no effect.
jkr: By "the COM loader" I mean the code within Windows that loads the DLL via 'LoadLibrary()' from the location specified in the registry.  From the DLL/COM Redirection patent at http://www.freepatentsonline.com/6976037.html :

[...] consider an application named "c:\myapp\myapp.exe" that calls a library loading routine ("LoadLibrary" in WINDOWS operating systems) using the pathname "c:\programfiles\commonfiles\system\mydll.dll." The directory ("c:\myapp") where the application resides is searched for a file named "myapp.exe.local." If that file is found, then the directory ("c:\myapp") is searched for an isolated, or local, version of "mydll.dll". If such a file is found in the directory, the library loading routine will load "c:\myapp\mydll.dll." Otherwise, the library loading routine will load the global version of the DLL/COM "c:\Windows\System\mydll.dll.

So the behaviour of LoadLibrary ought to match my expectations (whether called by the COM loader or not).  Bear in mind that LoadLibrary knows it is being called in the context of x.exe (or myapp.exe in the description from the patent).
(I love the fact that Microsoft's patents provide much clearer documentation than their actual documentation.  8-)
I know what you mean, but that does not apply. If the registry entry is

c:\Two\y.ocx

and 'LoadLibrary()' is called using an absolute path, this precedes over

c:\One\x.exe.local

Try it.

What you can do is make the .ocx modules a 'passthrough proxy' to your actual COM DLL.
jkr: "Try it."  OK, I've tried it, and it works the way I expect.  I have a directory C:\src\tests\dll-redir.  In there I have a DLL called dll.dll and an application called app.exe.  Under there I have a subdirectory called another-dir, in which is a copy of dll.dll.  The code for the app and the DLL look like this:

// cl app.c
#include "windows.h"
int main(void)
{
    HINSTANCE dll = LoadLibrary("C:\\src\\tests\\dll-redir\\another-dir\\dll.dll");
    return 0;
}

// cl /LD dll.c /link user32.lib
#include "windows.h"
BOOL WINAPI DllMain( HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved )
{
    char pathname[MAX_PATH];
    if ( fdwReason == DLL_PROCESS_ATTACH )
    {
        GetModuleFileName(hinstDLL, pathname, MAX_PATH);
        MessageBox(NULL, pathname, "Hello", MB_OK | MB_ICONINFORMATION);
    }
}

I run app.exe and it displays "C:\src\tests\dll-redir\another-dir\dll.dll".  I create C:\src\tests\dll-redir\app.exe.local and run app.exe, and it displays "C:\src\tests\dll-redir\dll.dll".

So giving a full pathname to LoadLibrary does *not* override a .local file (of course, or .local files would be useless).  Something else is preventing my app from loading the right OCX.

(The OCX is a third party control.  I can't modify it, and I don't want to have to make a whole new clone of its interface.)
ASKER CERTIFIED SOLUTION
Avatar of jkr
jkr
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
DependencyWalker shows it opening the local DLL, with no mention of the registered one.  Whatever it is that's making it open the registered one must be happening under DependencyWalker's radar.

However, I think FileMonitor may be lying, or the OS is doing something weird.  The FileMonitor output shows both the registered one and the local one being opened, and only the registered one being read into memory.  But when I change the DLLs to pop up a diagnostic message, it's the local one's message that appears.  The local one is being executed despite (according to FileMonitor) never being read...  I need to stop trusting the tools and do some more digging.
(Update: I'm looking sternly at LoadRegTypeLib, wondering whether it fails to support the .local system.  More follows...)
I think I have it.  http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsetup/html/sidebyside.asp says "Do not use the LoadRegTypeLib function to load type libraries through the registry."  My third party OCX is doing just that.  I need to do some more digging, but I suspect what is happening is that the control is being loaded initially by the COM loader, which goes via LoadLibrary and hence respects the .local file, but later on the control is accessing some piece of itself via LoadRegTypeLib, which is loading the registered version.

Looking at the loaded DLLs in Process Explorer I do indeed see two versions of the library loaded into the address space of the process.  And if I put a breakpoint in LoadRegTypeLib and step over it, that's the point where File Monitor tells me that the registered version is read from disk.

I need to either persuade LoadRegTypeLib to respect .local, or persuade the OCX not to use it (but I don't think I have the source code to that piece).
That was it.  LoadRegTypeLib doesn't respect .local files, and this OCX uses LoadRegTypeLib to access its own interfaces.  I've used API hooking to redirect the OCX's LoadRegTypeLib calls to another function that uses LoadTypeLib instead, and it all works.

I've accepted your pointer to DependencyWalker because it was that that put me on the right track, albeit indirectly.  Thanks!