Solved

SetWindowsHookEx problem?

Posted on 1999-01-18
49
434 Views
Last Modified: 2013-12-03
After loading the DLL which is used in conjunction with some (large) code segment, the following code freezes my computer:

if (!hhkGetMessage)
{
  HOOKPROC hp =            (HOOKPROC)GetProcAddress((HINSTANCE)hmodHook,                          "SpyGetMsgProc");

  if (!(hhkGetMessage = SetWindowsHookEx(WH_GETMESSAGE, hp,          (HINSTANCE)hmodHook, 0)))
  {
    DWORD err = GetLastError();
    return FALSE;
  }
}

THe only way to free up my computer is to CTL+ALT+DEL and kill the application and Visual C++.

At first I thought that it was not finding the function inside the DLL but it gets a valid address not far off the library address.

     Any ideas???

     Cheers
0
Comment
Question by:ross_nolan
  • 25
  • 23
49 Comments
 
LVL 86

Expert Comment

by:jkr
ID: 1418909
Does the hook proc get called?
Is the DLL with the hook proc the one with the large code segment? If the answer to the last Q is 'yes', you should consider to put the hook in its own DLL, as global hooks are mapped into _every_ GUI process' address space (which easily coul make the system hang ;-)
0
 

Author Comment

by:ross_nolan
ID: 1418910
THe hook procedure is in its own DLL.
sorry for not making this more clearer.

THe large code segment that went before was to set up an invisible window to communicate with the DLL.
So that when a message is intercepted by the DLL, in the system, the DLL would send the message to the invisible window which would handle it.

Unfortunitately I need the DLL as I want to be able to intercept all message in the system.

As for the HookProc this is within the DLL so I can only presume that it is called, as this DLL worked for a sample application which I found, but was not unfortunitely unsuitable for the purposes of my project

0
 
LVL 86

Expert Comment

by:jkr
ID: 1418911
>>So that when a message is intercepted by the DLL, in the
>>system, the DLL would send the message to the invisible window
>>which would handle it.
This seems to cause the problem (I hope you don't use 'SendMessage()' ;-) - as the hook is global, it also is used for your 'invisible' window, causing the system to enter a loop. You should make sure that you don't send messages to the invisible window that were already sent to it...

0
 

Author Comment

by:ross_nolan
ID: 1418912
Yes the DLL does use Send Message, but as I already have stated I got this DLL (which worked no problem) with another application.
I also used a lot of what I thought was the appropriate code from this application.

Any suggestions would be appriciated.
Also when I try to set up a link to the hook procedure in the DLL does this cause the procedure to call this procedure once for testing, or what is going on in the background for it to be sent a message to crash the system.

Cheers
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418913
Hmm, at first i suggest converting 'SendMessage()' to 'PostMessage()' - what' you're describing sounds like a neat 'deadlock', and i suspect 'SendMessage()' to be the cause...
0
 

Author Comment

by:ross_nolan
ID: 1418914
I will give that a go tomorrow as I am heading home soon

Thanks
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418915
Me too (9:30pm here ;-)
0
 
LVL 3

Expert Comment

by:luisr
ID: 1418916

I would recommend adding functions in the DLL that install (call SetWindowsHookEx from inside the DLL) and unistall the hook, and export those functions. Also save the HHOOK in a global variable and make it shared between all processes. Send me an e-mail if you want more information, but I'd recommend taking a look to chapter 6 (the Hooks chapter) of Windows 95: A Developer's Guide by Jeff Richter and Johnattan Locke (excellent book!)
0
 

Author Comment

by:ross_nolan
ID: 1418917
When I change SendMessage() to PostMessage() I get an error number "126"

Any idea what this is???
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418918
That's strange - it means 'specified module not found'...
But you're using a valid window handle, aren't you?
0
 

Author Comment

by:ross_nolan
ID: 1418919
The following is a copy of the HookProc contained in the DLL
(I know I am stating old ground but do you know why the DLL would work for one application and not for another)
THe PostMessage() is near the bottom.

Cheers

BOOL WINAPI
HookProc(
    HWND hwnd,
    UINT uiMessage,
    WPARAM wParam,
    LPARAM lParam
    )
{
    HWND hwndSpyingOn;
    HWND hwndSpyApp;

    if (ghwndSpyHook == NULL || !IsWindow(ghwndSpyHook))
    {
        //
        // Spy has terminated. Find the new window.
        //
        FindSpyWindow();
    }

    if (ghwndSpyHook != NULL && hwnd != ghwndSpyHook)
    {
        hwndSpyingOn = (HWND)GetWindowLong(ghwndSpyHook, 0);
        hwndSpyApp = (HWND)GetWindowLong(ghwndSpyHook, sizeof(HWND));

        //
        // Send the message on asynchronously for Spy to deal with if
        // it is the appropriate hwndSpyingOn window to spy on.
        //

        if (hwndSpyingOn == hwnd
            || (hwndSpyingOn == HWND_ALL && hwnd != hwndSpyApp
            && !IsChild(hwndSpyApp, hwnd)))
        {
            gsmd.wParam = wParam;
            gsmd.lParam = lParam;

            gcds.dwData = uiMessage;

            //SendMessage(ghwndSpyHook, WM_COPYDATA, (WPARAM)hwnd,                      (LPARAM)&gcds);
      PostMessage(ghwndSpyHook, WM_COPYDATA, (WPARAM)hwnd,                       (LPARAM)&gcds);

            return TRUE;
        }
    }

    return FALSE;
}

0
 
LVL 86

Expert Comment

by:jkr
ID: 1418920
Did you put the global variables into a shared data segment? If not, such weird error codes can easily appear ;-)
0
 

Author Comment

by:ross_nolan
ID: 1418921
What do you mean by a shared data segment???
Sorry but I am still getting used to programming in VC


0
 
LVL 86

Expert Comment

by:jkr
ID: 1418922
A shared data segmant means using a construct like this

#pragma data_seg( ".shared")
// global data goes here, _MUST_ be initialized!
HHOOK       g_hhk   =   NULL;
#pragma data_seg()

One other thing - you should also use a module definition file (.def) for your DLL, which is used to set the shared data segm.:
SECTIONS
shared READ WRITE SHARED

If you don't use a shared data segment, each process that uses the DLL will hold its own copy of the variable, and the contents will differ therefore from process to process.
0
 

Author Comment

by:ross_nolan
ID: 1418923
OK The (.DEF) solved the DLL running problem.

Now when the the PostMessage() procedure in the DLL is called it works, the only problem it is that the message is not received in the application.
I thought it might be a problem with the first parameter but the HWND has a valid value.

Any thoughts???
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418924
Hmm - what do you mean by 'valid value'? A window handle is only valid if 'IsWindow()' returns non-zero... You might want to try 'GetWindowText()' to find out the window title, this xould give some clarification. Another possibility would be that 'WM_COPYDATA' isn't handled by the target application...
0
 

Author Comment

by:ross_nolan
ID: 1418925
WM_COPYDATA is handled by the target application and I tried IsWindow in the DLL and it returned a non zero value

Any ideas???
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418926
Did you use MS Spy++ to check the message traffic?
0
 

Author Comment

by:ross_nolan
ID: 1418927
When I tried to use MS Spy++ the computer crashes and I have to reboot but when I don't run Spy++ I can step through the application no proplem.

Any ideas??
0
 

Author Comment

by:ross_nolan
ID: 1418928
Ì just had a thought.
Now don't get narkey and say congradulations, but what if it is the fact that the messages that the DLL is sending out are 'posted' as opposed to 'sent', would this make any significant difference.
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418929
It shouldn't - here's what the docs say:
'The SendMessage function sends the specified message to a window or windows. The function calls the window procedure for the specified window and does not return until the window procedure has processed the message. The PostMessage function, in contrast, posts a message to a thread’s message queue and returns immediately.'


0
 

Author Comment

by:ross_nolan
ID: 1418930
I have a feeling it is a problem with my GetMessage() function call in my invisible window.

any thoughts
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418931
Hmm - could you post the relevant code?
0
 

Author Comment

by:ross_nolan
ID: 1418932
Here is the code concerned:

The first function is called at application start up, and it creates a thread of the 2nd function. Any variables not defined in the function are global.

// Global variables
      HWND ghwndSpyApp;
      HANDLE ghHookThread;
      HWND ghwndSpyHook;
      HWND ghwndSpyingOn;
      BOOL gfSpyOn;
      BOOL gfSpyAll;
      HINSTANCE ghInst;

I get the value of the 'ghInst' variable through using the function AfxGetInstanceHandle( ) to get the HInstance of the main MFC application.

bool CreateHookThread(VOID)
{
      WNDCLASS wc;
      DWORD Id;

    //
    // Register a class for the hook stuff to forward its messages to.
    //
    wc.hCursor        = NULL;    // this window never shown, so no
    wc.hIcon          = NULL;    // cursor or icon are necessary
    wc.lpszMenuName   = NULL;
    wc.lpszClassName  = HOOKWINDOWCLASS;
    wc.hbrBackground  = (HBRUSH)(COLOR_WINDOW + 1);
    wc.hInstance      =      /*(HINSTANCE)*/ghInst;
    wc.style          = 0;
    wc.lpfnWndProc    = HookWndProc;
    wc.cbWndExtra     = sizeof(HWND) + sizeof(HWND);
    wc.cbClsExtra     = 0;

    if (!RegisterClass(&wc))
        return FALSE;

    //
    // Now create another thread to handle the new queue
    //
    if (!(ghHookThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)HookMain,
        0L, STANDARD_RIGHTS_REQUIRED, &Id)))
        return FALSE;

    return TRUE;
}


DWORD HookMain(LPVOID lpv)
{
          MSG msg;

    //
    // Create a hidden window for all to find, but not to see
    //
            ghwndSpyHook = CreateWindow(HOOKWINDOWCLASS, HOOKWINDOWNAME,
        WS_OVERLAPPEDWINDOW,
        0, 0, 0, 0,
        (HWND) NULL,        /* no parent */
        (HMENU) NULL,       /* use class menu */
        ghInst,    /* handle to window instance */
        (LPSTR) NULL        /* no params to pass on */
        );

    if (!ghwndSpyHook)
    {
            ExitThread(0);
    }

   // This function tells the DLL to monitor all windows
    SetWindowToSpyOn(HWND_ALL);

    //
    // Polling forwarded messages from hook's event queue
    //
    while (IsWindow(ghwndSpyHook) && GetMessage(&msg, ghwndSpyHook, 0, 0))
    {
        if (gfProcessHooks)
        {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
        }
    }

    ghwndSpyHook = NULL;


      return 0;
}


0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 86

Expert Comment

by:jkr
ID: 1418933
You left out the WindowProc ;-)
Is the 'gfProcessHooks' flag set to TRUE?
0
 

Author Comment

by:ross_nolan
ID: 1418934
Sorry about that, the 'gfProcessHooks' flag is set to TRUE.
Here is the code you were looking for:
I put a break point at the start of this function but when the DLL transmits its messages to the hidden application the break point is not encountered???


LRESULT CALLBACK HookWndProc(HWND hwnd, unsigned int msg, WPARAM wParam, LPARAM lParam)
{

    switch (msg)
    {
        //
        // New message for Win32 - allows the application to pass data to another application.
        //      
        case WM_COPYDATA:
            {
                MSG msgT;

                msgT.hwnd = (HWND)wParam;
                msgT.message = ((PCOPYDATASTRUCT)lParam)->dwData;
                msgT.wParam = ((PSPYMSGDATA)((PCOPYDATASTRUCT)lParam)->lpData)->wParam;
                msgT.lParam = ((PSPYMSGDATA)((PCOPYDATASTRUCT)lParam)->lpData)->lParam;
//DbgPrintf("S Received Message hwnd:%8.8x msg:%d", msgT.hwnd, msgT.message);
                        //PrintMsg(&msgT);
//DbgPrintf("S Printed Message hwnd:%8.8x msg:%d", msgT.hwnd, msgT.message);
                        CString str((char*)msgT.message);
//                        lb->AddString(str);
                  }

            return TRUE;

        case WM_CREATE:
            //
            // Initialize the second HWND in the window words to be the
            // window handle of the spy app.  This will be queried by
            // the hook DLL.
            //
            SetWindowLong(hwnd, sizeof(HWND), (LONG)ghwndSpyApp);
            return 0;

        case WM_DESTROY:
            PostQuitMessage(0);
            return 0;

        case WM_NCDESTROY:
            gfProcessHooks = FALSE;
            break;
    }

      return DefWindowProc(hwnd, msg, wParam, lParam);
}

Cheers
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418935
Hmm, i think i see the problem, try:
    //
    // Now create another thread to handle the new queue
    //
    if (!(ghHookThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)HookMain,
        0L, STANDARD_RIGHTS_REQUIRED, &Id)))
        return FALSE;

 if (!AttachThreadInput ( ld, GetCurrentThreadId(), TRUE))
      return( FALSE);
0
 

Author Comment

by:ross_nolan
ID: 1418936
This does not seem to have made a difference, but I will investigate it further tomorrow as I have to go now

Cheers
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418937
Try it - when your application already created a window, it's perhaps better to attach the new thread to it's input...
0
 

Author Comment

by:ross_nolan
ID: 1418938
Should I place the code in the HookMain function or the CreateHookThread function???

if (!AttachThreadInput ( ld, GetCurrentThreadId(), TRUE))
            return( FALSE
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418939
Well, you should put the code directly after the call to 'CreateThread()' - it would even better if you used 'CREATE_SUSPENDED' and call 'ResumeThread()' after 'AttachThreadInput()'...
0
 

Author Comment

by:ross_nolan
ID: 1418940
sorry but I get an error when I call AttachThreadInput()
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418941
Which one ;-) ?
0
 

Author Comment

by:ross_nolan
ID: 1418942
It appears to be not giving out the error now.

But still I am not stoping on the breakpoints in the HookWndProc() function in the WM_COPYDATA case.

0
 
LVL 86

Expert Comment

by:jkr
ID: 1418943
Ooops - i just read in the docs about 'WM_COPYDATA':
'An application must use the SendMessage function to send this message, not the PostMessage function.'
Well, this could lead into some trouble...
What about sending the message data through a mailslot/socket/named pipe/local RPC call rather than using 'WM_COPYDATA'?
0
 

Author Comment

by:ross_nolan
ID: 1418944
eh!!!
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418945
???? ;-)
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418946
Well, as it isn't a lot of work, you should give it a try again using 'SendMessage()' (i wouldn't recommend this unless the docs didn't state anything else...)
On NT named pipes would be the best solution, but mailslots are easy too handle...
0
 

Author Comment

by:ross_nolan
ID: 1418947
I think I understand what you are saying, it is going to take me a short while to read up on them but I think a named pipe is the way to go.

I will post again by Monday when I should know if this can be done.

Cheers

0
 
LVL 86

Expert Comment

by:jkr
ID: 1418948
One thing you should know before trying this: Win9x can only serve as a named pipe _client_, this could be a big obstacle...
0
 

Author Comment

by:ross_nolan
ID: 1418949
I have found a Win32 application that gets messages from the system message queue, and I will be tweeking it over the next few days to transmit bitmap information to a MFC client.

Cheers
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418950
Tell me how it's working - and also if you think i may lock the Q ;-)
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418951
And - how is it working?
0
 

Author Comment

by:ross_nolan
ID: 1418952
It's going all right but I am at the coment trying to integrate a MFC class into the win32 application.
I have being trying to get in tough with the guy who gave me the code but he has not yet responded to my e-mail.

Any idea would be most appreciated!!!
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418953
>>trying to integrate a MFC class into the win32 application

I'd not recommend to do so, as using a system wide hook will cause _all_ MFC DLLs to be loaded into _every_ process' address space...
0
 

Author Comment

by:ross_nolan
ID: 1418954
I now realize the futility of this exercise
0
 
LVL 86

Expert Comment

by:jkr
ID: 1418955
Well, Ross, what's going to happen with this Q?
0
 

Author Comment

by:ross_nolan
ID: 1418956
At the moment I am putting any MFC code into a regular DLL and exporting functions for the Win32 program to use.
0
 
LVL 86

Accepted Solution

by:
jkr earned 100 total points
ID: 1418957
Well, I'm finally going to answer this before it's deleted.

BTW: Take a look at 'http://www.codeguru.com/win32/ipcdemo.shtml', "Inter-Process Communication Using WM_COPYDATA "
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

This article shows a few slightly more advanced techniques for Windows 7 gadget programming, including how to save and restore user settings for your gadget and how to populate the "details" panel that is displayed in the Windows 7 gadget gallery.  …
This article describes how to programmatically preset the "Pages per Sheet" option that's available with most printer drivers.   This setting lets you do "n-Up" printing, where two, four, or more pages are printed on each sheet of paper. If your …
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now