Link to home
Start Free TrialLog in
Avatar of vivin
vivin

asked on

DllRegisterServer failed . Return code was 0x80020009

I am attempting to reverse engineer legacy application. Let me give you a little bit of a background. We have a completely working installation but there is a disconnect between the working version and the source code that we got. I am able to successfully compile the source code using MS VC++ 6.0 and use Wise Installation Studio 7.0 to create .msi. The installer is suppose to self register dll's but these dll's fail as a result of which the registry does not have certain CLSID's. The application does not work due to missing registry keys (CLSID's).

When I use the command prompt (as an administrator or not) and try to do

regsvr32 somedll.dll

I get the following error

DllRegisterServer in somedll.dll failed . Return code was 0x80020009

This is how my DllRegisterServer method looks like

/////////////////////////////////////////////////////////////////////////////
// DllRegisterServer - Adds entries to the system registry

STDAPI DllRegisterServer(void)
{
#ifdef _MERGE_PROXYSTUB
    HRESULT hRes = PrxDllRegisterServer();
    if (FAILED(hRes)) {
        return hRes;
 }
#endif
    // registers object, typelib and all interfaces in typelib
    return _Module.RegisterServer(TRUE);
}

The debugger enters assembly code after call to _Module.RegisterServer(TRUE); And some where in middle of executing "_Module.RegisterServer" function - it bails out and throws the following message

DllRegisterServer in somedll.dll failed . Return code was 0x80020009


When I look at the assembly code - here are some details

97:   #ifdef _MERGE_PROXYSTUB
98:       HRESULT hRes = PrxDllRegisterServer();
10003FC8   call        @ILT+1490(_PrxDllRegisterServer@0) (100015d7)
10003FCD   mov         dword ptr [ebp-4],eax
99:       if (FAILED(hRes)) {
10003FD0   cmp         dword ptr [ebp-4],0
10003FD4   jge         DllRegisterServer+2Bh (10003fdb)
100:          return hRes;
10003FD6   mov         eax,dword ptr [ebp-4]
10003FD9   jmp         DllRegisterServer+39h (10003fe9)
101:      }
102:  #endif
103:      // registers object, typelib and all interfaces in typelib
104:      return _Module.RegisterServer(TRUE);
10003FDB   push        0
10003FDD   push        1
10003FDF   mov         ecx,offset _Module (1022e568)
10003FE4   call        @ILT+715(ATL::CComModule::RegisterServer) (100012d0)

What caught my eye was "ATL" (ATL::CComModule::RegisterServer). I have been reading about ATL on some forums and the compatibility issues it might have. Does it ring any bells ?


I also thought of using an alternate route :
I have the original "somedll.dll" with me (the one that registers and works) and I have the new compiled "somedll.dll" with me (the one that does not register). I am using dependency walker to find out the dependencies between the two. To a larget extent both dlls are similar except for the order.

For instance with the orginal dll the sequence is

.....
.....
ATL.DLL
somedll.dll
.....
.....




While with the one that does not work the sequnce is

......
......
somedll.dll
.....
.....
.....
ATL.DLL
.....
.....


Does the "ordering" matter ? The reason I am coming back to ATL.DLL is because -  that is the point where debugger bombs  as pointed out earlier.

I am in a fix and need to resolve this issue ASAP. I will really appreciate all your help. Please let me know if you have any further questions. I am running XP, so things like turning UAC off and other permission related (run as administrator) recommendations are not useful.

Thank you
Viv
Avatar of DanRollins
DanRollins
Flag of United States of America image

is _MERGE_PROXYSTUB #defined?  Was it #defined in the earlier version?
On thing you can try is changing the settings to 'Use MFC in a static library"
 
ASKER CERTIFIED SOLUTION
Avatar of vivin
vivin

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial