Solved

"Screen Scraping" the contents displayed by a window

Posted on 2004-08-01
21
570 Views
Last Modified: 2008-01-09
Hi,

This problem seems to be hard.  I've tried to find a solution on my own, but without luck.

There is an application running on Windows XP that displays various graphics.  I know nothing about it.  The only thing I have is the window's handle.

I want to be able to analyse the graphics displayed in the window.  It would be nice for example if I could call a function called getBitMap(myWindowsHandle) and have returned an object or the data which is the bitmap of the window contents.  This data must exist somewhere if Windows is displaying it, right?

Thank you

0
Comment
Question by:Gezna
  • 13
  • 5
  • 2
  • +1
21 Comments
 
LVL 55

Expert Comment

by:Jaime Olivares
Comment Utility
You can use the WinAPI PrintWindow() function: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/prntspol_6qpj.asp
- Create your own dc.
- Create a compatible bitmap with screen size
- Select bitmap into your dc using SelectObject()
- Invoke PrintWindow function specifying window's handle and your dc handle
- Now your bitmap contain what window displays, you can save or process it.
0
 

Author Comment

by:Gezna
Comment Utility
That sounds like a good answer.  Is this something I'd have to implement in C or C++?   That's okay if it is as it shouldn't take too long to figure out how to program in either language.  I have a computer science degree.
0
 
LVL 37

Expert Comment

by:gregoryyoung
Comment Utility
what language would you like to do it in ? I have examples in C# and vb.net, pretty much any language can call windows apis ...
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
Get the size of the source window:

      RECT rectSrc;
      GetClientRect(hwndSrc, &rectSrc);
      int nWidth, nHeight;
      nWidth = rectSrc.right;
      nHeight = rectSrc.bottom;

Get a DC to the source window:

      HDC hdcSrc;
      hdcSrc = GetDC(hwndSrc);

Create an area of memory to receive the image bits:

      BITMAPINFO bmi;
      // Prepare DIB format
      ZeroMemory(&bmi, sizeof(bmi));
      bmi.bmiHeader.biSize = sizeof(bmi.bmiHeader);
      bmi.bmiHeader.biWidth = nWidth;
      bmi.bmiHeader.biHeight = -nHeight;
      bmi.bmiHeader.biPlanes = 1;
      bmi.bmiHeader.biBitCount = 32;
      bmi.bmiHeader.biCompression = BI_RGB;

      DWORD *pDibBits;
      pDibBits = 0;

      HBITMAP hDib;
      hDib = CreateDIBSection(hdcScrn, &bmi, DIB_RGB_COLORS, (void**)&pDibBits, NULL, 0);

      // Windows provides the memory for the image and puts a pointer to it in pDibBits.

Create a destination DC:

      HDC hdcDst;
      hdcDst = CreateCompatibleDC(hdcSrc);

Select the DIB into the memory device context:

      HGDIOBJ hOldDst;
      hOldDst = SelectObject(hdcDst, hDib);

BitBlt (copy) the image to your DIB section:

      BitBlt(hdcDst, 0, 0, nWidth, nHeight, hdcSrc, 0, 0, SRCCOPY);

Clean up:

      SelectObject(hdcDst, hOldDst);
      DeleteDC(hdcDst);
      ReleaseDC(hwndSrc, hdcSrc);

At this point, pDibBits points to an array of 32-bit values, one per pixel, in BGRx format (the first byte is the blue value for the 0th pixel, the second byte is the green value for the 0th pixel, the third byte is the red value for the 0th pixel, the fourth byte is unused. The fifth byte is the blue value for the second pixel, etc...)

When you are finished with the image data, clean it up:

      DeleteObject(hDib);
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
Actually, I just noticed something in your question:

>> This data must exist somewhere if Windows is displaying it, right?

Wrong. Windows sends a message to the app whenever an area of the window needs to be updated. The app then draws any needed area of the screen on demand. See WM_PAINT and InvalidateRect at MSDN for details. The image DOES exist in video memory, and IF THE WINDOW IS VISIBLE, my code above will get the content of it.
0
 
LVL 55

Expert Comment

by:Jaime Olivares
Comment Utility
PrintWindow() will capture any window content, even if hidden. (If you have the proper HWND)
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
PrintWindow is only supported on XP and Windows Server 2003
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
PrintWindow only replaces the BitBlt in my code. You still have to setup the destination memory device context.
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
Looks like you can send the WM_PRINT message to the window, passing in prepared destination device context from my code (again, replacing only the BitBlt). Looks somewhat equivalent to PrintWindow, but portable all the way back to Win95:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/pantdraw_3qwk.asp

You'd replace the BitBlt with:

SendMessage(hwndSrc, WM_PRINT, hdcDst, (LPARAM)(PRF_CHILDREN | PRF_CLIENT | PRF_OWNED);

This enables it to work even when the window is not visible. It sends a message to the owner of the source window (as I described about WM_PAINT) and apparently captures its drawing into the device context passed with WM_PRINT.
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
Actually...

SendMessage(hwndSrc, WM_PRINT, (WPARAM)hdcDst, (LPARAM)(PRF_CHILDREN | PRF_CLIENT | PRF_OWNED));

And if you use this, you can drop the preparation and cleanup of the source (in my code). Remove:

GetDC(hwndSrc)

and

ReleaseDC(hwndSrc, hdcSrc)
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 

Author Comment

by:Gezna
Comment Utility
Thank you very much.  I will try to implement it in C++.  Since it is a new language to me, it might take a while before I can get it to work and award the points.  I just finished writing a hello world app and I will award the points when I succeed in getting the bitmap.  I will award to whoever had the most useful information.
0
 

Author Comment

by:Gezna
Comment Utility
adg,

I've put your code into my C++ program, and it compiles just fine.  I'd like to display the bitmap just to make sure your code is actually working.  Can you show me how I'd display the bitmap that is in pDibBits?

0
 
LVL 8

Accepted Solution

by:
adg080898 earned 500 total points
Comment Utility
In short, you select the dib into a device context, then use that device context as the source to BitBlt the image to the destination device context. You can create the source device context using CreateCompatibleDC. See WndProc_Paint below.

Here is a program which draws the window under the cursor.

// grabwin.cpp

#include <windows.h>

// This information is shared to draw
// the (captured) window under the mouse

HWND hwndMain;
DWORD *pDibBits;
HBITMAP hDib;
int nImgWidth, nImgHeight;

LRESULT WndProc_Paint(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
      HDC hdcPaint;
      PAINTSTRUCT ps;

      if (!hDib) {
            BeginPaint(hwnd, &ps);
            EndPaint(hwnd, &ps);
            return 0;
      }

      // Prepare to draw and get area that needs updating
      hdcPaint = BeginPaint(hwnd, &ps);

      // Prepare to render the DIB bits
      HDC hdcSrc = CreateCompatibleDC(hdcPaint);
      HGDIOBJ hOld = SelectObject(hdcSrc, hDib);

      // Blt just the area requested by the paint message
      BitBlt(hdcPaint,
                  ps.rcPaint.left, ps.rcPaint.top,
                  ps.rcPaint.right - ps.rcPaint.left,
                  ps.rcPaint.bottom - ps.rcPaint.top,
                  hdcSrc, ps.rcPaint.left, ps.rcPaint.top,
                  SRCCOPY);

      // If the area extends past the bitmap, clear the area
      if (ps.rcPaint.right > nImgWidth) {
            // Clear right edge
            BitBlt(hdcPaint, nImgWidth, 0,
                        ps.rcPaint.right - nImgWidth, ps.rcPaint.bottom,
                        0, 0, 0, WHITENESS);
      }
      // If the area extends past the bitmap, clear the area
      if (ps.rcPaint.bottom > nImgHeight) {
            BitBlt(hdcPaint, ps.rcPaint.left, nImgHeight,
                        ps.rcPaint.right - ps.rcPaint.left,
                        ps.rcPaint.bottom - nImgHeight,
                        0, 0, 0, WHITENESS);
      }

      // Eject the bitmap from the context
      SelectObject(hdcPaint, hOld);
      hOld = 0;

      // Clean up
      DeleteDC(hdcSrc);
      hdcSrc = 0;
      EndPaint(hwnd, &ps);
      hdcPaint = 0;

      return TRUE;
}

LRESULT CALLBACK WndProc_Main(HWND hwnd, UINT nMsg,
            WPARAM wParam, LPARAM lParam)
{
      switch (nMsg) {
      case WM_CREATE:
            hwndMain = hwnd;
            return 0;

      case WM_DESTROY:
            if (hDib) {
                  DeleteObject(hDib);
                  hDib = 0;
                  pDibBits = 0;
                  nImgWidth = 0;
                  nImgHeight = 0;
            }

            return 0;

      case WM_PAINT:
            return WndProc_Paint(hwnd, wParam, lParam);

      case WM_CLOSE:
            PostQuitMessage(0);
            return 0;

      case WM_NCDESTROY:
            hwndMain = 0;
            return 0;

      }

      return DefWindowProc(hwnd, nMsg, wParam, lParam);
}

void DoGrab()
{
      HWND hwndSrc;
      POINT pntMse;

      // Clean up
      if (hDib) {
            DeleteObject(hDib);
            hDib = 0;
            pDibBits = 0;
            nImgWidth = 0;
            nImgHeight = 0;
      }

      GetCursorPos(&pntMse);
      hwndSrc = WindowFromPoint(pntMse);

      // Get the size of the source window:
      RECT rectSrc;
      GetClientRect(hwndSrc, &rectSrc);
      nImgWidth = rectSrc.right;
      nImgHeight = rectSrc.bottom;

      // Get a DC to the source window:

      HDC hdcSrc;
      hdcSrc = GetDC(hwndSrc);

      // Create an area of memory to receive the image bits:

      BITMAPINFO bmi;
      // Prepare DIB format
      ZeroMemory(&bmi, sizeof(bmi));
      bmi.bmiHeader.biSize = sizeof(bmi.bmiHeader);
      bmi.bmiHeader.biWidth = nImgWidth;
      bmi.bmiHeader.biHeight = -nImgHeight;
      bmi.bmiHeader.biPlanes = 1;
      bmi.bmiHeader.biBitCount = 32;
      bmi.bmiHeader.biCompression = BI_RGB;

      hDib = CreateDIBSection(hdcSrc, &bmi, DIB_RGB_COLORS, (void**)&pDibBits, NULL, 0);

      // Windows provides the memory for the image and puts a pointer to it in pDibBits.

      // Create a destination DC:

      HDC hdcDst;
      hdcDst = CreateCompatibleDC(hdcSrc);

      // Select the DIB into the memory device context:

      HGDIOBJ hOldDst;
      hOldDst = SelectObject(hdcDst, hDib);

      // BitBlt (copy) the image to your DIB section:

#define USE_WM_PAINT 0
#if USE_WM_PAINT
      SendMessage(hwndSrc, WM_PRINT, (WPARAM)hdcDst,
                  (LPARAM)(0 & (PRF_CHILDREN | PRF_CLIENT | PRF_OWNED)));
#else
      BitBlt(hdcDst, 0, 0, nImgWidth, nImgHeight, hdcSrc, 0, 0, SRCCOPY);
#endif

      // Clean up:

      SelectObject(hdcDst, hOldDst);
      hOldDst = 0;
      DeleteDC(hdcDst);
      hdcDst = 0;
      ReleaseDC(hwndSrc, hdcSrc);
      hdcSrc = 0;

      // Redraw
      InvalidateRect(hwndMain, NULL, FALSE);
}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
            LPSTR lpCmdLine, int nCmdShow)
{
      WNDCLASS wc;
      MSG msg;

      ZeroMemory(&wc, sizeof(wc));
      wc.lpszClassName = "ShowGrabber";
      wc.lpfnWndProc = WndProc_Main;
      wc.hInstance = hInstance;

      RegisterClass(&wc);
      CreateWindowEx(0, "ShowGrabber", "Grabber",
                  WS_OVERLAPPEDWINDOW,
                  CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT,
                  0, 0, hInstance, 0);

      ShowWindow(hwndMain, SW_SHOWDEFAULT);

      ZeroMemory(&msg, sizeof(msg));
      while (hwndMain) {
            // Dispatch messages until queue empty
            while (PeekMessage(&msg, 0, 0, 0, PM_REMOVE)) {
                  TranslateMessage(&msg);
                  DispatchMessage(&msg);

                  if (msg.message == WM_QUIT)
                        break;
            }

            if (msg.message == WM_QUIT)
                  break;

            // Idle time
            // Do another grab and sleep for 1/4 second to prevent
            // pegging cpu usage
            DoGrab();

            Sleep(250);
      }

      if (hwndMain)
            DestroyWindow(hwndMain);

      return 0;
}

0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
The nicest thing about using CreateDIBSection is you have direct access to the pixel data. You can modify the data pointed to by pDibBits and it changes the image. Each pixel is a 32-bit value. It is BGRx format, meaning bit 0 thru 7 are blue, bit 8-15 are green, and bit 16-23 are red, the high 8 bits are unused.
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
You can always select the DIB handle into a device context and draw on it like any other bitmap.
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
You'll note that in my program there is a #define to select using WM_PAINT to paint the image. I could not get it to work, and I suspect that the window needs to support the message for it to work. There is also a WM_PAINTCLIENT message but that didn't work either. I played with the flags passed in LPARAM but WM_PAINT, and WM_PAINTCLIENT just got me a black window. I could not find the definition for PaintWindow anywhere in my SDK headers, so I was unable to try that as an alternative. My example code is working using the BitBlt method I originally suggested. It takes a raw snapshot of the area of that window, getting whatever might be on top of that window.

0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
oops, not "WM_PAINT, and WM_PAINTCLIENT", I mean "WM_PRINT, and WM_PRINTCLIENT"
0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
I just noticed a bug in WndProc_Paint. Here is the correction:

LRESULT WndProc_Paint(HWND hwnd, WPARAM wParam, LPARAM lParam)
{
     HDC hdcPaint;
     PAINTSTRUCT ps;

     if (!hDib) {
          BeginPaint(hwnd, &ps);
          EndPaint(hwnd, &ps);
          return 0;
     }

     // Prepare to draw and get area that needs updating
     hdcPaint = BeginPaint(hwnd, &ps);

     // Prepare to render the DIB bits
     HDC hdcSrc = CreateCompatibleDC(hdcPaint);
     HGDIOBJ hOld = SelectObject(hdcSrc, hDib);

     // Blt just the area requested by the paint message
     BitBlt(hdcPaint,
               ps.rcPaint.left, ps.rcPaint.top,
               ps.rcPaint.right - ps.rcPaint.left,
               ps.rcPaint.bottom - ps.rcPaint.top,
               hdcSrc, ps.rcPaint.left, ps.rcPaint.top,
               SRCCOPY);

     // If the area extends past the bitmap, clear the area
     if (ps.rcPaint.right > nImgWidth) {
          // Clear right edge
          BitBlt(hdcPaint, nImgWidth, 0,
                    ps.rcPaint.right - nImgWidth, ps.rcPaint.bottom,
                    0, 0, 0, WHITENESS);
     }
     // If the area extends past the bitmap, clear the area
     if (ps.rcPaint.bottom > nImgHeight) {
          BitBlt(hdcPaint, ps.rcPaint.left, nImgHeight,
                    ps.rcPaint.right - ps.rcPaint.left,
                    ps.rcPaint.bottom - nImgHeight,
                    0, 0, 0, WHITENESS);
     }

     // Clean up
     SelectObject(hdcSrc, hOld);
     hOld = 0;
     DeleteDC(hdcSrc);
     hdcSrc = 0;
     EndPaint(hwnd, &ps);
     hdcPaint = 0;

     return TRUE;
}
0
 

Author Comment

by:Gezna
Comment Utility
Boy! this C++ stuff is a lot harder than Java.  I'm really struggling with this.  I have a C++ textbook and there's nothing on graphics.  

I am confident I'll get it eventually.  If there's anything you can say to make it easier on me I'd welcome that.  If you think you're explaining your code as clearly as possilbe then just let me figure it out.  I'll get it, eventually.

0
 
LVL 8

Expert Comment

by:adg080898
Comment Utility
This is GDI programming. It might help to look at Windows GDI:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/wingdistart_9ezp.asp

Painting and drawing (preparing to draw to an output device like screen or printer):
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdi/pantdraw_3wtj.asp

Did you compile and run the program I posted? You want to draw the captured DIB, so I will describe what is necessary to draw a DIB.

If you want to programmatically examine/process the image pixels, you just look at the data at pDibBits (in my example, from CreateDIBSection).

To draw a DIB, you create a source memory device context, from which you will copy the image. You will then select the DIB into the source device context.

The destination device context is from GetDC or BeginPaint, or it might even be a printer device context.

You want the source and destination device contexts to be compatible, so you create the source device context from the destination.

You copy the image from the source device context to the destination device context. To copy the image, you might use BitBlt or StretchBlt.

Normally in windows, drawing the window is event driven. You never know when something will popup on top of your app. When that popup disappears, windows now has a "hole" in the display where it needs its old content restored. Windows sends a WM_PAINT message to the program and the program draws the area specified in the PAINTSTRUCT returned from BeginPaint. You should do all your drawing in the WM_PAINT handler, other parts of the program should just use InvalidateRect to cause an appropriate WM_PAINT message to occur. You can force a WM_PAINT using UpdateWindow.
The other destination you will be using is a printer device context. You can use a printer context to draw the DIB on a page.
If you really just want to draw it immediately without being event driven, you can use GetDC to get a device context for a window.

So you have a destination device context, hdcDest. If I want to draw the DIB hDib on the hdcDest:

HDC hdcSrc;
hdcSrc = CreateCompatibleDC(hdcDest);

Now, I select the DIB into the source device context, but when you select something into a DC, you have to promise to select the old thing back in when you are done, the old thing is returned from SelectObject.

HGDIOBJ hOldSrc;
hOldSrc = SelectObject(hdcSrc, hDib);

Now I have a source context all setup to draw the DIB from, I just do the copy from the source to the destination:

BitBlt(hdcDest, 0, 0, nWidth, nHeight, hdcSrc, 0, 0, SRCCOPY);

I am now done with the source device context, so let's tear it down and remove it from existence.

SelectObject(hdcSrc, hOldSrc);
DeleteDC(hdcSrc);

That is how to draw a DIB from a DIB handle. Is that what you wanted, or are you asking how to examine the pixels in the image programmatically?
0
 

Author Comment

by:Gezna
Comment Utility
Thankyou I pasted your program into Visual C++ and after a few false starts I managed to get it to compile and executing.  It works.  My goal is to eventually start analysing the pixel data programmatically, but we'll save that for another day!
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Displaying an arrayList in a listView using the default adapter is rarely the best solution. To get full control of your display data, and to be able to refresh it after editing, requires the use of a custom adapter.
Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now