Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 464
  • Last Modified:

SetWindowsHookEx problem?

After loading the DLL which is used in conjunction with some (large) code segment, the following code freezes my computer:

if (!hhkGetMessage)
{
  HOOKPROC hp =            (HOOKPROC)GetProcAddress((HINSTANCE)hmodHook,                          "SpyGetMsgProc");

  if (!(hhkGetMessage = SetWindowsHookEx(WH_GETMESSAGE, hp,          (HINSTANCE)hmodHook, 0)))
  {
    DWORD err = GetLastError();
    return FALSE;
  }
}

THe only way to free up my computer is to CTL+ALT+DEL and kill the application and Visual C++.

At first I thought that it was not finding the function inside the DLL but it gets a valid address not far off the library address.

     Any ideas???

     Cheers
0
ross_nolan
Asked:
ross_nolan
  • 25
  • 23
1 Solution
 
jkrCommented:
Does the hook proc get called?
Is the DLL with the hook proc the one with the large code segment? If the answer to the last Q is 'yes', you should consider to put the hook in its own DLL, as global hooks are mapped into _every_ GUI process' address space (which easily coul make the system hang ;-)
0
 
ross_nolanAuthor Commented:
THe hook procedure is in its own DLL.
sorry for not making this more clearer.

THe large code segment that went before was to set up an invisible window to communicate with the DLL.
So that when a message is intercepted by the DLL, in the system, the DLL would send the message to the invisible window which would handle it.

Unfortunitately I need the DLL as I want to be able to intercept all message in the system.

As for the HookProc this is within the DLL so I can only presume that it is called, as this DLL worked for a sample application which I found, but was not unfortunitely unsuitable for the purposes of my project

0
 
jkrCommented:
>>So that when a message is intercepted by the DLL, in the
>>system, the DLL would send the message to the invisible window
>>which would handle it.
This seems to cause the problem (I hope you don't use 'SendMessage()' ;-) - as the hook is global, it also is used for your 'invisible' window, causing the system to enter a loop. You should make sure that you don't send messages to the invisible window that were already sent to it...

0
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

 
ross_nolanAuthor Commented:
Yes the DLL does use Send Message, but as I already have stated I got this DLL (which worked no problem) with another application.
I also used a lot of what I thought was the appropriate code from this application.

Any suggestions would be appriciated.
Also when I try to set up a link to the hook procedure in the DLL does this cause the procedure to call this procedure once for testing, or what is going on in the background for it to be sent a message to crash the system.

Cheers
0
 
jkrCommented:
Hmm, at first i suggest converting 'SendMessage()' to 'PostMessage()' - what' you're describing sounds like a neat 'deadlock', and i suspect 'SendMessage()' to be the cause...
0
 
ross_nolanAuthor Commented:
I will give that a go tomorrow as I am heading home soon

Thanks
0
 
jkrCommented:
Me too (9:30pm here ;-)
0
 
luisrCommented:

I would recommend adding functions in the DLL that install (call SetWindowsHookEx from inside the DLL) and unistall the hook, and export those functions. Also save the HHOOK in a global variable and make it shared between all processes. Send me an e-mail if you want more information, but I'd recommend taking a look to chapter 6 (the Hooks chapter) of Windows 95: A Developer's Guide by Jeff Richter and Johnattan Locke (excellent book!)
0
 
ross_nolanAuthor Commented:
When I change SendMessage() to PostMessage() I get an error number "126"

Any idea what this is???
0
 
jkrCommented:
That's strange - it means 'specified module not found'...
But you're using a valid window handle, aren't you?
0
 
ross_nolanAuthor Commented:
The following is a copy of the HookProc contained in the DLL
(I know I am stating old ground but do you know why the DLL would work for one application and not for another)
THe PostMessage() is near the bottom.

Cheers

BOOL WINAPI
HookProc(
    HWND hwnd,
    UINT uiMessage,
    WPARAM wParam,
    LPARAM lParam
    )
{
    HWND hwndSpyingOn;
    HWND hwndSpyApp;

    if (ghwndSpyHook == NULL || !IsWindow(ghwndSpyHook))
    {
        //
        // Spy has terminated. Find the new window.
        //
        FindSpyWindow();
    }

    if (ghwndSpyHook != NULL && hwnd != ghwndSpyHook)
    {
        hwndSpyingOn = (HWND)GetWindowLong(ghwndSpyHook, 0);
        hwndSpyApp = (HWND)GetWindowLong(ghwndSpyHook, sizeof(HWND));

        //
        // Send the message on asynchronously for Spy to deal with if
        // it is the appropriate hwndSpyingOn window to spy on.
        //

        if (hwndSpyingOn == hwnd
            || (hwndSpyingOn == HWND_ALL && hwnd != hwndSpyApp
            && !IsChild(hwndSpyApp, hwnd)))
        {
            gsmd.wParam = wParam;
            gsmd.lParam = lParam;

            gcds.dwData = uiMessage;

            //SendMessage(ghwndSpyHook, WM_COPYDATA, (WPARAM)hwnd,                      (LPARAM)&gcds);
      PostMessage(ghwndSpyHook, WM_COPYDATA, (WPARAM)hwnd,                       (LPARAM)&gcds);

            return TRUE;
        }
    }

    return FALSE;
}

0
 
jkrCommented:
Did you put the global variables into a shared data segment? If not, such weird error codes can easily appear ;-)
0
 
ross_nolanAuthor Commented:
What do you mean by a shared data segment???
Sorry but I am still getting used to programming in VC


0
 
jkrCommented:
A shared data segmant means using a construct like this

#pragma data_seg( ".shared")
// global data goes here, _MUST_ be initialized!
HHOOK       g_hhk   =   NULL;
#pragma data_seg()

One other thing - you should also use a module definition file (.def) for your DLL, which is used to set the shared data segm.:
SECTIONS
shared READ WRITE SHARED

If you don't use a shared data segment, each process that uses the DLL will hold its own copy of the variable, and the contents will differ therefore from process to process.
0
 
ross_nolanAuthor Commented:
OK The (.DEF) solved the DLL running problem.

Now when the the PostMessage() procedure in the DLL is called it works, the only problem it is that the message is not received in the application.
I thought it might be a problem with the first parameter but the HWND has a valid value.

Any thoughts???
0
 
jkrCommented:
Hmm - what do you mean by 'valid value'? A window handle is only valid if 'IsWindow()' returns non-zero... You might want to try 'GetWindowText()' to find out the window title, this xould give some clarification. Another possibility would be that 'WM_COPYDATA' isn't handled by the target application...
0
 
ross_nolanAuthor Commented:
WM_COPYDATA is handled by the target application and I tried IsWindow in the DLL and it returned a non zero value

Any ideas???
0
 
jkrCommented:
Did you use MS Spy++ to check the message traffic?
0
 
ross_nolanAuthor Commented:
When I tried to use MS Spy++ the computer crashes and I have to reboot but when I don't run Spy++ I can step through the application no proplem.

Any ideas??
0
 
ross_nolanAuthor Commented:
Ì just had a thought.
Now don't get narkey and say congradulations, but what if it is the fact that the messages that the DLL is sending out are 'posted' as opposed to 'sent', would this make any significant difference.
0
 
jkrCommented:
It shouldn't - here's what the docs say:
'The SendMessage function sends the specified message to a window or windows. The function calls the window procedure for the specified window and does not return until the window procedure has processed the message. The PostMessage function, in contrast, posts a message to a thread’s message queue and returns immediately.'


0
 
ross_nolanAuthor Commented:
I have a feeling it is a problem with my GetMessage() function call in my invisible window.

any thoughts
0
 
jkrCommented:
Hmm - could you post the relevant code?
0
 
ross_nolanAuthor Commented:
Here is the code concerned:

The first function is called at application start up, and it creates a thread of the 2nd function. Any variables not defined in the function are global.

// Global variables
      HWND ghwndSpyApp;
      HANDLE ghHookThread;
      HWND ghwndSpyHook;
      HWND ghwndSpyingOn;
      BOOL gfSpyOn;
      BOOL gfSpyAll;
      HINSTANCE ghInst;

I get the value of the 'ghInst' variable through using the function AfxGetInstanceHandle( ) to get the HInstance of the main MFC application.

bool CreateHookThread(VOID)
{
      WNDCLASS wc;
      DWORD Id;

    //
    // Register a class for the hook stuff to forward its messages to.
    //
    wc.hCursor        = NULL;    // this window never shown, so no
    wc.hIcon          = NULL;    // cursor or icon are necessary
    wc.lpszMenuName   = NULL;
    wc.lpszClassName  = HOOKWINDOWCLASS;
    wc.hbrBackground  = (HBRUSH)(COLOR_WINDOW + 1);
    wc.hInstance      =      /*(HINSTANCE)*/ghInst;
    wc.style          = 0;
    wc.lpfnWndProc    = HookWndProc;
    wc.cbWndExtra     = sizeof(HWND) + sizeof(HWND);
    wc.cbClsExtra     = 0;

    if (!RegisterClass(&wc))
        return FALSE;

    //
    // Now create another thread to handle the new queue
    //
    if (!(ghHookThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)HookMain,
        0L, STANDARD_RIGHTS_REQUIRED, &Id)))
        return FALSE;

    return TRUE;
}


DWORD HookMain(LPVOID lpv)
{
          MSG msg;

    //
    // Create a hidden window for all to find, but not to see
    //
            ghwndSpyHook = CreateWindow(HOOKWINDOWCLASS, HOOKWINDOWNAME,
        WS_OVERLAPPEDWINDOW,
        0, 0, 0, 0,
        (HWND) NULL,        /* no parent */
        (HMENU) NULL,       /* use class menu */
        ghInst,    /* handle to window instance */
        (LPSTR) NULL        /* no params to pass on */
        );

    if (!ghwndSpyHook)
    {
            ExitThread(0);
    }

   // This function tells the DLL to monitor all windows
    SetWindowToSpyOn(HWND_ALL);

    //
    // Polling forwarded messages from hook's event queue
    //
    while (IsWindow(ghwndSpyHook) && GetMessage(&msg, ghwndSpyHook, 0, 0))
    {
        if (gfProcessHooks)
        {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
        }
    }

    ghwndSpyHook = NULL;


      return 0;
}


0
 
jkrCommented:
You left out the WindowProc ;-)
Is the 'gfProcessHooks' flag set to TRUE?
0
 
ross_nolanAuthor Commented:
Sorry about that, the 'gfProcessHooks' flag is set to TRUE.
Here is the code you were looking for:
I put a break point at the start of this function but when the DLL transmits its messages to the hidden application the break point is not encountered???


LRESULT CALLBACK HookWndProc(HWND hwnd, unsigned int msg, WPARAM wParam, LPARAM lParam)
{

    switch (msg)
    {
        //
        // New message for Win32 - allows the application to pass data to another application.
        //      
        case WM_COPYDATA:
            {
                MSG msgT;

                msgT.hwnd = (HWND)wParam;
                msgT.message = ((PCOPYDATASTRUCT)lParam)->dwData;
                msgT.wParam = ((PSPYMSGDATA)((PCOPYDATASTRUCT)lParam)->lpData)->wParam;
                msgT.lParam = ((PSPYMSGDATA)((PCOPYDATASTRUCT)lParam)->lpData)->lParam;
//DbgPrintf("S Received Message hwnd:%8.8x msg:%d", msgT.hwnd, msgT.message);
                        //PrintMsg(&msgT);
//DbgPrintf("S Printed Message hwnd:%8.8x msg:%d", msgT.hwnd, msgT.message);
                        CString str((char*)msgT.message);
//                        lb->AddString(str);
                  }

            return TRUE;

        case WM_CREATE:
            //
            // Initialize the second HWND in the window words to be the
            // window handle of the spy app.  This will be queried by
            // the hook DLL.
            //
            SetWindowLong(hwnd, sizeof(HWND), (LONG)ghwndSpyApp);
            return 0;

        case WM_DESTROY:
            PostQuitMessage(0);
            return 0;

        case WM_NCDESTROY:
            gfProcessHooks = FALSE;
            break;
    }

      return DefWindowProc(hwnd, msg, wParam, lParam);
}

Cheers
0
 
jkrCommented:
Hmm, i think i see the problem, try:
    //
    // Now create another thread to handle the new queue
    //
    if (!(ghHookThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)HookMain,
        0L, STANDARD_RIGHTS_REQUIRED, &Id)))
        return FALSE;

 if (!AttachThreadInput ( ld, GetCurrentThreadId(), TRUE))
      return( FALSE);
0
 
ross_nolanAuthor Commented:
This does not seem to have made a difference, but I will investigate it further tomorrow as I have to go now

Cheers
0
 
jkrCommented:
Try it - when your application already created a window, it's perhaps better to attach the new thread to it's input...
0
 
ross_nolanAuthor Commented:
Should I place the code in the HookMain function or the CreateHookThread function???

if (!AttachThreadInput ( ld, GetCurrentThreadId(), TRUE))
            return( FALSE
0
 
jkrCommented:
Well, you should put the code directly after the call to 'CreateThread()' - it would even better if you used 'CREATE_SUSPENDED' and call 'ResumeThread()' after 'AttachThreadInput()'...
0
 
ross_nolanAuthor Commented:
sorry but I get an error when I call AttachThreadInput()
0
 
jkrCommented:
Which one ;-) ?
0
 
ross_nolanAuthor Commented:
It appears to be not giving out the error now.

But still I am not stoping on the breakpoints in the HookWndProc() function in the WM_COPYDATA case.

0
 
jkrCommented:
Ooops - i just read in the docs about 'WM_COPYDATA':
'An application must use the SendMessage function to send this message, not the PostMessage function.'
Well, this could lead into some trouble...
What about sending the message data through a mailslot/socket/named pipe/local RPC call rather than using 'WM_COPYDATA'?
0
 
ross_nolanAuthor Commented:
eh!!!
0
 
jkrCommented:
???? ;-)
0
 
jkrCommented:
Well, as it isn't a lot of work, you should give it a try again using 'SendMessage()' (i wouldn't recommend this unless the docs didn't state anything else...)
On NT named pipes would be the best solution, but mailslots are easy too handle...
0
 
ross_nolanAuthor Commented:
I think I understand what you are saying, it is going to take me a short while to read up on them but I think a named pipe is the way to go.

I will post again by Monday when I should know if this can be done.

Cheers

0
 
jkrCommented:
One thing you should know before trying this: Win9x can only serve as a named pipe _client_, this could be a big obstacle...
0
 
ross_nolanAuthor Commented:
I have found a Win32 application that gets messages from the system message queue, and I will be tweeking it over the next few days to transmit bitmap information to a MFC client.

Cheers
0
 
jkrCommented:
Tell me how it's working - and also if you think i may lock the Q ;-)
0
 
jkrCommented:
And - how is it working?
0
 
ross_nolanAuthor Commented:
It's going all right but I am at the coment trying to integrate a MFC class into the win32 application.
I have being trying to get in tough with the guy who gave me the code but he has not yet responded to my e-mail.

Any idea would be most appreciated!!!
0
 
jkrCommented:
>>trying to integrate a MFC class into the win32 application

I'd not recommend to do so, as using a system wide hook will cause _all_ MFC DLLs to be loaded into _every_ process' address space...
0
 
ross_nolanAuthor Commented:
I now realize the futility of this exercise
0
 
jkrCommented:
Well, Ross, what's going to happen with this Q?
0
 
ross_nolanAuthor Commented:
At the moment I am putting any MFC code into a regular DLL and exporting functions for the Win32 program to use.
0
 
jkrCommented:
Well, I'm finally going to answer this before it's deleted.

BTW: Take a look at 'http://www.codeguru.com/win32/ipcdemo.shtml', "Inter-Process Communication Using WM_COPYDATA "
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 25
  • 23
Tackle projects and never again get stuck behind a technical roadblock.
Join Now