• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 654
  • Last Modified:

"Screen Scraping" the contents displayed by a window

Hi,

This problem seems to be hard.  I've tried to find a solution on my own, but without luck.

There is an application running on Windows XP that displays various graphics.  I know nothing about it.  The only thing I have is the window's handle.

I want to be able to analyse the graphics displayed in the window.  It would be nice for example if I could call a function called getBitMap(myWindowsHandle) and have returned an object or the data which is the bitmap of the window contents.  This data must exist somewhere if Windows is displaying it, right?

Thank you

0
Gezna
Asked:
Gezna
  • 13
  • 5
  • 2
  • +1
1 Solution
 
Jaime OlivaresSoftware ArchitectCommented:
You can use the WinAPI PrintWindow() function: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/prntspol_6qpj.asp
- Create your own dc.
- Create a compatible bitmap with screen size
- Select bitmap into your dc using SelectObject()
- Invoke PrintWindow function specifying window's handle and your dc handle
- Now your bitmap contain what window displays, you can save or process it.
0
 
GeznaAuthor Commented:
That sounds like a good answer.  Is this something I'd have to implement in C or C++?   That's okay if it is as it shouldn't take too long to figure out how to program in either language.  I have a computer science degree.
0
 
gregoryyoungCommented:
what language would you like to do it in ? I have examples in C# and vb.net, pretty much any language can call windows apis ...
0
Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

 
adg080898Commented:
Get the size of the source window:

      RECT rectSrc;
      GetClientRect(hwndSrc, &rectSrc);
      int nWidth, nHeight;
      nWidth = rectSrc.right;
      nHeight = rectSrc.bottom;

Get a DC to the source window:

      HDC hdcSrc;
      hdcSrc = GetDC(hwndSrc);

Create an area of memory to receive the image bits:

      BITMAPINFO bmi;
      // Prepare DIB format
      ZeroMemory(&bmi, sizeof(bmi));
      bmi.bmiHeader.biSize = sizeof(bmi.bmiHeader);
      bmi.bmiHeader.biWidth = nWidth;
      bmi.bmiHeader.biHeight = -nHeight;
      bmi.bmiHeader.biPlanes = 1;
      bmi.bmiHeader.biBitCount = 32;
      bmi.bmiHeader.biCompression = BI_RGB;

      DWORD *pDibBits;
      pDibBits = 0;

      HBITMAP hDib;
      hDib = CreateDIBSection(hdcScrn, &bmi, DIB_RGB_COLORS, (void**)&pDibBits, NULL, 0);

      // Windows provides the memory for the image and puts a pointer to it in pDibBits.

Create a destination DC:

      HDC hdcDst;
      hdcDst = CreateCompatibleDC(hdcSrc);

Select the DIB into the memory device context:

      HGDIOBJ hOldDst;
      hOldDst = SelectObject(hdcDst, hDib);

BitBlt (copy) the image to your DIB section:

      BitBlt(hdcDst, 0, 0, nWidth, nHeight, hdcSrc, 0, 0, SRCCOPY);

Clean up:

      SelectObject(hdcDst, hOldDst);
      DeleteDC(hdcDst);
      ReleaseDC(hwndSrc, hdcSrc);

At this point, pDibBits points to an array of 32-bit values, one per pixel, in BGRx format (the first byte is the blue value for the 0th pixel, the second byte is the green value for the 0th pixel, the third byte is the red value for the 0th pixel, the fourth byte is unused. The fifth byte is the blue value for the second pixel, etc...)

When you are finished with the image data, clean it up:

      DeleteObject(hDib);
0
 
adg080898Commented:
Actually, I just noticed something in your question:

>> This data must exist somewhere if Windows is displaying it, right?

Wrong. Windows sends a message to the app whenever an area of the window needs to be updated. The app then draws any needed area of the screen on demand. See WM_PAINT and InvalidateRect at MSDN for details. The image DOES exist in video memory, and IF THE WINDOW IS VISIBLE, my code above will get the content of it.
0
 
Jaime OlivaresSoftware ArchitectCommented:
PrintWindow() will capture any window content, even if hidden. (If you have the proper HWND)
0
 
adg080898Commented:
PrintWindow is only supported on XP and Windows Server 2003
0
 
adg080898Commented:
PrintWindow only replaces the BitBlt in my code. You still have to setup the destination memory device context.
0
 
adg080898Commented:
Looks like you can send the WM_PRINT message to the window, passing in prepared destination device context from my code (again, replacing only the BitBlt). Looks somewhat equivalent to PrintWindow, but portable all the way back to Win95:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/pantdraw_3qwk.asp

You'd replace the BitBlt with:

SendMessage(hwndSrc, WM_PRINT, hdcDst, (LPARAM)(PRF_CHILDREN | PRF_CLIENT | PRF_OWNED);

This enables it to work even when the window is not visible. It sends a message to the owner of the source window (as I described about WM_PAINT) and apparently captures its drawing into the device context passed with WM_PRINT.
0
 
adg080898Commented:
Actually...

SendMessage(hwndSrc, WM_PRINT, (WPARAM)hdcDst, (LPARAM)(PRF_CHILDREN | PRF_CLIENT | PRF_OWNED));

And if you use this, you can drop the preparation and cleanup of the source (in my code). Remove:

GetDC(hwndSrc)

and

ReleaseDC(hwndSrc, hdcSrc)
0
 
GeznaAuthor Commented:
Thank you very much.  I will try to implement it in C++.  Since it is a new language to me, it might take a while before I can get it to work and award the points.  I just finished writing a hello world app and I will award the points when I succeed in getting the bitmap.  I will award to whoever had the most useful information.
0
 
GeznaAuthor Commented:
adg,

I've put your code into my C++ program, and it compiles just fine.  I'd like to display the bitmap just to make sure your code is actually working.  Can you show me how I'd display the bitmap that is in pDibBits?

0
 
adg080898Commented:
In short, you select the dib into a device context, then use that device context as the source to BitBlt the image to the destination device context. You can create the source device context using CreateCompatibleDC. See WndProc_Paint below.

Here is a program which draws the window under the cursor.

// grabwin.cpp

#include <windows.h>

// This information is shared to draw
// the (captured) window under the mouse

HWND hwndMain;
DWORD *pDibBits;
HBITMAP hDib;
int nImgWidth, nImgHeight;

LRESULT WndProc_Paint(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
      HDC hdcPaint;
      PAINTSTRUCT ps;

      if (!hDib) {
            BeginPaint(hwnd, &ps);
            EndPaint(hwnd, &ps);
            return 0;
      }

      // Prepare to draw and get area that needs updating
      hdcPaint = BeginPaint(hwnd, &ps);

      // Prepare to render the DIB bits
      HDC hdcSrc = CreateCompatibleDC(hdcPaint);
      HGDIOBJ hOld = SelectObject(hdcSrc, hDib);

      // Blt just the area requested by the paint message
      BitBlt(hdcPaint,
                  ps.rcPaint.left, ps.rcPaint.top,
                  ps.rcPaint.right - ps.rcPaint.left,
                  ps.rcPaint.bottom - ps.rcPaint.top,
                  hdcSrc, ps.rcPaint.left, ps.rcPaint.top,
                  SRCCOPY);

      // If the area extends past the bitmap, clear the area
      if (ps.rcPaint.right > nImgWidth) {
            // Clear right edge
            BitBlt(hdcPaint, nImgWidth, 0,
                        ps.rcPaint.right - nImgWidth, ps.rcPaint.bottom,
                        0, 0, 0, WHITENESS);
      }
      // If the area extends past the bitmap, clear the area
      if (ps.rcPaint.bottom > nImgHeight) {
            BitBlt(hdcPaint, ps.rcPaint.left, nImgHeight,
                        ps.rcPaint.right - ps.rcPaint.left,
                        ps.rcPaint.bottom - nImgHeight,
                        0, 0, 0, WHITENESS);
      }

      // Eject the bitmap from the context
      SelectObject(hdcPaint, hOld);
      hOld = 0;

      // Clean up
      DeleteDC(hdcSrc);
      hdcSrc = 0;
      EndPaint(hwnd, &ps);
      hdcPaint = 0;

      return TRUE;
}

LRESULT CALLBACK WndProc_Main(HWND hwnd, UINT nMsg,
            WPARAM wParam, LPARAM lParam)
{
      switch (nMsg) {
      case WM_CREATE:
            hwndMain = hwnd;
            return 0;

      case WM_DESTROY:
            if (hDib) {
                  DeleteObject(hDib);
                  hDib = 0;
                  pDibBits = 0;
                  nImgWidth = 0;
                  nImgHeight = 0;
            }

            return 0;

      case WM_PAINT:
            return WndProc_Paint(hwnd, wParam, lParam);

      case WM_CLOSE:
            PostQuitMessage(0);
            return 0;

      case WM_NCDESTROY:
            hwndMain = 0;
            return 0;

      }

      return DefWindowProc(hwnd, nMsg, wParam, lParam);
}

void DoGrab()
{
      HWND hwndSrc;
      POINT pntMse;

      // Clean up
      if (hDib) {
            DeleteObject(hDib);
            hDib = 0;
            pDibBits = 0;
            nImgWidth = 0;
            nImgHeight = 0;
      }

      GetCursorPos(&pntMse);
      hwndSrc = WindowFromPoint(pntMse);

      // Get the size of the source window:
      RECT rectSrc;
      GetClientRect(hwndSrc, &rectSrc);
      nImgWidth = rectSrc.right;
      nImgHeight = rectSrc.bottom;

      // Get a DC to the source window:

      HDC hdcSrc;
      hdcSrc = GetDC(hwndSrc);

      // Create an area of memory to receive the image bits:

      BITMAPINFO bmi;
      // Prepare DIB format
      ZeroMemory(&bmi, sizeof(bmi));
      bmi.bmiHeader.biSize = sizeof(bmi.bmiHeader);
      bmi.bmiHeader.biWidth = nImgWidth;
      bmi.bmiHeader.biHeight = -nImgHeight;
      bmi.bmiHeader.biPlanes = 1;
      bmi.bmiHeader.biBitCount = 32;
      bmi.bmiHeader.biCompression = BI_RGB;

      hDib = CreateDIBSection(hdcSrc, &bmi, DIB_RGB_COLORS, (void**)&pDibBits, NULL, 0);

      // Windows provides the memory for the image and puts a pointer to it in pDibBits.

      // Create a destination DC:

      HDC hdcDst;
      hdcDst = CreateCompatibleDC(hdcSrc);

      // Select the DIB into the memory device context:

      HGDIOBJ hOldDst;
      hOldDst = SelectObject(hdcDst, hDib);

      // BitBlt (copy) the image to your DIB section:

#define USE_WM_PAINT 0
#if USE_WM_PAINT
      SendMessage(hwndSrc, WM_PRINT, (WPARAM)hdcDst,
                  (LPARAM)(0 & (PRF_CHILDREN | PRF_CLIENT | PRF_OWNED)));
#else
      BitBlt(hdcDst, 0, 0, nImgWidth, nImgHeight, hdcSrc, 0, 0, SRCCOPY);
#endif

      // Clean up:

      SelectObject(hdcDst, hOldDst);
      hOldDst = 0;
      DeleteDC(hdcDst);
      hdcDst = 0;
      ReleaseDC(hwndSrc, hdcSrc);
      hdcSrc = 0;

      // Redraw
      InvalidateRect(hwndMain, NULL, FALSE);
}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
            LPSTR lpCmdLine, int nCmdShow)
{
      WNDCLASS wc;
      MSG msg;

      ZeroMemory(&wc, sizeof(wc));
      wc.lpszClassName = "ShowGrabber";
      wc.lpfnWndProc = WndProc_Main;
      wc.hInstance = hInstance;

      RegisterClass(&wc);
      CreateWindowEx(0, "ShowGrabber", "Grabber",
                  WS_OVERLAPPEDWINDOW,
                  CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT,
                  0, 0, hInstance, 0);

      ShowWindow(hwndMain, SW_SHOWDEFAULT);

      ZeroMemory(&msg, sizeof(msg));
      while (hwndMain) {
            // Dispatch messages until queue empty
            while (PeekMessage(&msg, 0, 0, 0, PM_REMOVE)) {
                  TranslateMessage(&msg);
                  DispatchMessage(&msg);

                  if (msg.message == WM_QUIT)
                        break;
            }

            if (msg.message == WM_QUIT)
                  break;

            // Idle time
            // Do another grab and sleep for 1/4 second to prevent
            // pegging cpu usage
            DoGrab();

            Sleep(250);
      }

      if (hwndMain)
            DestroyWindow(hwndMain);

      return 0;
}

0
 
adg080898Commented:
The nicest thing about using CreateDIBSection is you have direct access to the pixel data. You can modify the data pointed to by pDibBits and it changes the image. Each pixel is a 32-bit value. It is BGRx format, meaning bit 0 thru 7 are blue, bit 8-15 are green, and bit 16-23 are red, the high 8 bits are unused.
0
 
adg080898Commented:
You can always select the DIB handle into a device context and draw on it like any other bitmap.
0
 
adg080898Commented:
You'll note that in my program there is a #define to select using WM_PAINT to paint the image. I could not get it to work, and I suspect that the window needs to support the message for it to work. There is also a WM_PAINTCLIENT message but that didn't work either. I played with the flags passed in LPARAM but WM_PAINT, and WM_PAINTCLIENT just got me a black window. I could not find the definition for PaintWindow anywhere in my SDK headers, so I was unable to try that as an alternative. My example code is working using the BitBlt method I originally suggested. It takes a raw snapshot of the area of that window, getting whatever might be on top of that window.

0
 
adg080898Commented:
oops, not "WM_PAINT, and WM_PAINTCLIENT", I mean "WM_PRINT, and WM_PRINTCLIENT"
0
 
adg080898Commented:
I just noticed a bug in WndProc_Paint. Here is the correction:

LRESULT WndProc_Paint(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
     HDC hdcPaint;
     PAINTSTRUCT ps;

     if (!hDib) {
          BeginPaint(hwnd, &ps);
          EndPaint(hwnd, &ps);
          return 0;
     }

     // Prepare to draw and get area that needs updating
     hdcPaint = BeginPaint(hwnd, &ps);

     // Prepare to render the DIB bits
     HDC hdcSrc = CreateCompatibleDC(hdcPaint);
     HGDIOBJ hOld = SelectObject(hdcSrc, hDib);

     // Blt just the area requested by the paint message
     BitBlt(hdcPaint,
               ps.rcPaint.left, ps.rcPaint.top,
               ps.rcPaint.right - ps.rcPaint.left,
               ps.rcPaint.bottom - ps.rcPaint.top,
               hdcSrc, ps.rcPaint.left, ps.rcPaint.top,
               SRCCOPY);

     // If the area extends past the bitmap, clear the area
     if (ps.rcPaint.right > nImgWidth) {
          // Clear right edge
          BitBlt(hdcPaint, nImgWidth, 0,
                    ps.rcPaint.right - nImgWidth, ps.rcPaint.bottom,
                    0, 0, 0, WHITENESS);
     }
     // If the area extends past the bitmap, clear the area
     if (ps.rcPaint.bottom > nImgHeight) {
          BitBlt(hdcPaint, ps.rcPaint.left, nImgHeight,
                    ps.rcPaint.right - ps.rcPaint.left,
                    ps.rcPaint.bottom - nImgHeight,
                    0, 0, 0, WHITENESS);
     }

     // Clean up
     SelectObject(hdcSrc, hOld);
     hOld = 0;
     DeleteDC(hdcSrc);
     hdcSrc = 0;
     EndPaint(hwnd, &ps);
     hdcPaint = 0;

     return TRUE;
}
0
 
GeznaAuthor Commented:
Boy! this C++ stuff is a lot harder than Java.  I'm really struggling with this.  I have a C++ textbook and there's nothing on graphics.  

I am confident I'll get it eventually.  If there's anything you can say to make it easier on me I'd welcome that.  If you think you're explaining your code as clearly as possilbe then just let me figure it out.  I'll get it, eventually.

0
 
adg080898Commented:
This is GDI programming. It might help to look at Windows GDI:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/wingdistart_9ezp.asp

Painting and drawing (preparing to draw to an output device like screen or printer):
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/pantdraw_3wtj.asp

Did you compile and run the program I posted? You want to draw the captured DIB, so I will describe what is necessary to draw a DIB.

If you want to programmatically examine/process the image pixels, you just look at the data at pDibBits (in my example, from CreateDIBSection).

To draw a DIB, you create a source memory device context, from which you will copy the image. You will then select the DIB into the source device context.

The destination device context is from GetDC or BeginPaint, or it might even be a printer device context.

You want the source and destination device contexts to be compatible, so you create the source device context from the destination.

You copy the image from the source device context to the destination device context. To copy the image, you might use BitBlt or StretchBlt.

Normally in windows, drawing the window is event driven. You never know when something will popup on top of your app. When that popup disappears, windows now has a "hole" in the display where it needs its old content restored. Windows sends a WM_PAINT message to the program and the program draws the area specified in the PAINTSTRUCT returned from BeginPaint. You should do all your drawing in the WM_PAINT handler, other parts of the program should just use InvalidateRect to cause an appropriate WM_PAINT message to occur. You can force a WM_PAINT using UpdateWindow.
The other destination you will be using is a printer device context. You can use a printer context to draw the DIB on a page.
If you really just want to draw it immediately without being event driven, you can use GetDC to get a device context for a window.

So you have a destination device context, hdcDest. If I want to draw the DIB hDib on the hdcDest:

HDC hdcSrc;
hdcSrc = CreateCompatibleDC(hdcDest);

Now, I select the DIB into the source device context, but when you select something into a DC, you have to promise to select the old thing back in when you are done, the old thing is returned from SelectObject.

HGDIOBJ hOldSrc;
hOldSrc = SelectObject(hdcSrc, hDib);

Now I have a source context all setup to draw the DIB from, I just do the copy from the source to the destination:

BitBlt(hdcDest, 0, 0, nWidth, nHeight, hdcSrc, 0, 0, SRCCOPY);

I am now done with the source device context, so let's tear it down and remove it from existence.

SelectObject(hdcSrc, hOldSrc);
DeleteDC(hdcSrc);

That is how to draw a DIB from a DIB handle. Is that what you wanted, or are you asking how to examine the pixels in the image programmatically?
0
 
GeznaAuthor Commented:
Thankyou I pasted your program into Visual C++ and after a few false starts I managed to get it to compile and executing.  It works.  My goal is to eventually start analysing the pixel data programmatically, but we'll save that for another day!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

  • 13
  • 5
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now