subclassing procedure

Hello.

After reading a bit about hooks (Richter, etc.), I am trying to implement
a program that will subclass the procedure belonging to a dialog of another
application (on a Win95 or NT system).

Unfortunately, I cannot make it work.
Can you please help me find what's wrong?

What I'm trying to do is:

In my EXE file (console app):
------------------------------
1) Start the target application using CreateProcess()

2) Load a DLL with the hook using LoadLibrary() and
   get  the hooking procedure address using GetProcAddress().

3) Call the hooking procedure with the target app's thread ID as a parameter.

4) Wait for the target app to exit (using WaitForSingleObject())

5) Remove the hook, free the DLL, cleanup and exit.

No message loop or anything fancy...


In my DLL:
------------
1) In the DllMain() - Save the DLL hInstance.

2) In the hooking proc - call:
   hHook = SetWindowsHookEx(WH_GETMESSAGE, GetMsgProc, hinstDll, dwThreadId);
   passing the saved DLL hInstance and the target app's thread ID
   (from the EXE step #3).

   [problem #1]

3) In the Hook - do something like:

   if (code == HC_ACTION && wParam == PM_REMOVE &&
       ((MSG*)lParam)->message == WM_INITDIALOG &&
       MyCompareDialog(((MSG*)lParam)->hwnd))
   {
       oldDlgProc = SubclassDialog(((MSG*)lParam)->hwnd, MyDlgProc);
   }
   return CallNextHookEx(hHook, code, wParam, lParam);

   [problem #2]

4) In MyCompareDialog() - get the dialog's title using GetWindowText()
   and compare it to the title of the dialog I wish to subclass.

5) In MyDlgProc() - do something like:

   switch (message)
   {
       case WM_COMMAND:
           switch GET_WM_COMMAND_ID(wParam, lParam)
           {
               case 57670: // A button identifier, found with Spy++
                   MessageBox(NULL, "Gotcha!", "Info", MB_ICONINFORMATION);
                   return TRUE; // Message was processed
           }
   }
   // Call previous dialog procedure
   return CallWindowProc((WNDPROC)oldDlgProc, hWnd, message, wParam, lParam);

Now the problems:

Problem #1:
        The call to SetWindowsHookEx() sometimes fails with error 87
        (invalid parameter), but if I put a breakpoint on the line and
        then continue, it seems to work OK.  Huh?

Problem #2:
        The hook does not seem to catch WM_INITDIALOG.
        In fact, the first message it gets is WM_MOUSEMOVE (0x200).

zviorAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sapekCommented:
1. Make sure you set DllMain() the entry point for your dll. Otherwise your DllMain() would never be excuted and you wouldn't have rigth hInstance.2. Try using CBT hook instead. This what Microsoft originaly did in ctl3d.dll that subclassed dialog controls to give them 3d look.Adam
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Development

From novice to tech pro — start learning today.