Solved

Capture Screen Content or Window Content

Posted on 2007-11-14
24
3,992 Views
Last Modified: 2012-05-05
Using the examples I found earlier on EE I am able to capture the content of the screen. How ever, the only thing I see is a copy of my VBA source code, and not the "Other" window that I need to examine.

I need some help in understand how this part of windows works.

  1. Is there a 'Collection' of windows somewhere and if so, how do I reference it and get the count of windows.

  2. How do I identify each of the windows in the collection to find the one I need? Is there text or something else that I can examine to determine if I found the window I need.
 
 3. After finding the desired window, is there a map or directory that has specifics info. on the client controls (frames, textboxes, images, etc) that appear on the screen.

4. How can I get the content of these controls to variables in my VBA code. Specifically the content of an .bmp and .jpg image.
 
0
Comment
Question by:PropertyTaxAnalysts
  • 11
  • 10
  • 3
24 Comments
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20280814
It would help if you would show us the code you are currently attempting to use...

Also, is the "Other" window part of YOUR app?...or an EXTERNAL app?

The answers to your questions are dependent upon the environment you are working in.  If you are using VB6 and the window is part of your app then the things in question are relatively easy.  VBA doesn't keep a Forms collection as far as I know though.

If you are talking about external windows then it becomes fairly complicated as you must Windows APIs to enumerate all the windows.  There isn't a direct method to get a count.  Windows can be indentifed by their caption and/or ClassName.  It is possible to enumerate over the controls in a specific window and determine their classname and/or text.

So please give us more details about your environment and specifically what you are tyring to do...
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20281786
Idle Mind,

Thanks for your quick response. I appoligize for leaving out so me prerequisite details. I'm working in Excel VBA and I have programitically started Internet Explorer, navigated to the correct web site, and waited for the server to populate the IE window.  From my VBA code I need to find the IE window and extract a .bmp image from the screen.

The testing that I've done (sample below) so far only shows a window of my executing code. It looks like it only handles my task, and nothing else that may be in another window on the screen.  

I don't need a count as long as I can find the IE window that I need.

Sample Code
'=============================================================================
Function ScreenToClipBoard()
Dim ActiveHwnd As Long
Dim DeskHwnd As Long
Dim ForegroundHwnd As Long
Dim hdc As Long
Dim hdcMem As Long
Dim rect As RECT_Type
Dim junk As Long
Dim fwidth As Long, fheight As Long
Dim hBitmap As Long
 
    '---------------------------------------------------
    ' Get window handle to Windows and Microsoft Access
    '---------------------------------------------------
    DeskHwnd = GetDesktopWindow()
    ActiveHwnd = GetActiveWindow()
    ForegroundHwnd = GetForegroundWindow()

    '---------------------------------------------------
    ' Get screen coordinates of Active Window
    '---------------------------------------------------
    Call GetWindowRect(ActiveHwnd, rect)
    ' or
    ' Call GetWindowRect(ForegroundHwnd, rect)
    fwidth = rect.right - rect.left
    fheight = rect.bottom - rect.top
 
    '---------------------------------------------------
    ' Get the device context of Desktop and allocate memory
    '---------------------------------------------------
    hdc = GetDC(DeskHwnd)
    hdcMem = CreateCompatibleDC(hdc)
    hBitmap = CreateCompatibleBitmap(hdc, fwidth, fheight)
 
    If hBitmap <> 0 Then
       junk = SelectObject(hdcMem, hBitmap)
 
       '---------------------------------------------
       ' Copy the Desktop bitmap to memory location
       ' based on Microsoft Access coordinates.
       '---------------------------------------------
       junk = BitBlt(hdcMem, 0, 0, fwidth, fheight, hdc, rect.left, _
                     rect.top, SRCCOPY)
 
       '---------------------------------------------
       ' Set up the Clipboard and copy bitmap
       '---------------------------------------------
       junk = OpenClipboard(DeskHwnd)
       junk = EmptyClipboard()
       junk = SetClipboardData(CF_BITMAP, hBitmap)
       junk = CloseClipboard()
    End If
 
    '---------------------------------------------
    ' Clean up handles
    '---------------------------------------------
    junk = DeleteDC(hdcMem)
    junk = ReleaseDC(DeskHwnd, hdc)
 
End Function
'=============================================================================
I hope that I've answered your questions, but if I still have omitted something you need, I'll try to respond quickly.

John
0
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20281824
Ok...so you're using the ScreenToClipBoard() method...but how are you calling it?

The code is grabbing the coordinates of the Active window:

    Call GetWindowRect(ActiveHwnd, rect)

So make sure you are not stealing focus before callling the code...

Also, how are you starting Internet Explorer?  Depending on the method, we may be able to get a handle to that window directly...
0
Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20285869
Idle_Mind:

Below is the code I use from within my VBA program to bring up the Inernet Explorer. It also came from one of the EE contributors.

'=====================================================================
    Set IE = CreateObject("InternetExplorer.Application")
    Set objshell = CreateObject("WScript.Shell")

    With IE
        .left = 20
        .top = 20
        .Height = 500
        .Width = 950
        .MenuBar = True         ' Must be on since the 'Refresh' key may be used
        .Toolbar = True
        .StatusBar = True
        .navigate "http://www.google.com"
        .Visible = True
    End With
'================================================================

I'm using this because it sure seems to work, but if there is a better way, I'm all ears. I do have a need to extract and set HTML values (mainly strings) in the IE HTML from within my VBA code. So far that works also.  The .bmp and . jpg images I need are in popups and I haven't been able to find the variable name for these images within the HTML code. Please remember that I'm a real novice at the VBA programming and I barely qualify as a beginner for the HTML, but I'm eager to learn more about both of these areas.

I think I understand your comment about stealing the focus. When my VBA code executes the "IE.Navigate" command, IE becomes the active window, and when IE has completed my request, I become the active window again. If this is true, my code should only be able to capture the active window of itself. What can I do to circumvent this situation so I do not steal the focus from IE.

I hope this answers you questions.

John


   
       
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20291169
Hi John,

It does sound like you are having difficulty finding the correct window.  I don't think you can rely on either the Active Window or the Foreground Window.  What you could use is the windows API called FindWindow  to find the window you want.  You can either pass the exact windows title for that window, or, you can pass the "window class" to FindWindow to get the window handle.  If the window you spawn with the call to google always has the same title, you can use that.  

Otherwise, to get the window class of a window, you can use a tool such as SPY.  You probably have SPY listed as a tool under your Visual Studio Tools menu, but if not, google it and you can probaby download it free.  When you run SPY, you can move the cursor over any window on your screen and then click the window you're interested in and it will tell you the window class.  You can then use that in the FindWindow call to get a handle to that particular type of window.  Now, it might not work very well if you happen to have two or more IE browsers open on the screen at the same time as both will have the same window class.

Another thing you can check is to see if your object variable, IE has a property called hwnd, some of the VB/VBA object models include the window handle as a property.

Finally, once you navigate to the HTML page in your VBA code as above, the IE object should have some property that contains the HTML that was returned (might be InnerHTML, or Document, or Body -- don't really remember).  It would be possible for you to parse the HTML and find the reference in the HTML to the image's source.  The HTML would look something like:

<img src="http://www.bla.com/somepic.jpg"

I believe that you can then use LoadPicture to load the picture from its location into a picturebox or bitmap variable in your program.  That way, you're not relying on grabbing it off the screen, but rather getting it from its source.  If LoadPicture doesn't work, then maybe using a file stream object would.  You might search EE for Loading Bitmap from Internet or something like that.  I know I've participated in those threads before.

Good luck!
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20298885
mdougan,

I followed your suggestion and after the call to "FindWindow" the value of x is not zero.

Is the value of x the Hwnd value?

If so, I need to get the bitmap from the screen since its source seems to change the value every time I reference it. It's somewhat like a non repeatable serial number and the value on the screen is what I need.

Is there a map or directory that has specifics info. on the content of this window (frames, textboxes, images, etc), or do I need to develop some sort of "Pattern Matching" code that will enable me to find an anchor point of the .bmp image (x,y value of the top left corner).

After I find this point, what's the best way to examine the contents between the upper left anchor to the bottom right anchor so I can determine its content. Fortunately, the .bmp I need seems to always be in a rectangular shape.

Any suggested reading references for the Hwnd or DC controls/methods would be greatly appreciated since my brain does not yet have a firm concept/understanding of the tools/functions that I need to use to accomplish this objective.

Thanks for your suggestion and I feel that I've moved ahead by a Giant step.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20299906
Yes, the result of the FindWindow call is the window handle  (hWnd).

What you are trying to do is a very tricky thing.  It is "possible" that any text entry area, or image display area might have a name or handle that you can use to capture its contents.  Your best bet is to use that tool SPY, mouse over the image and/or click on the image in the browser and see if you get different values than when you mouse-over/click on any other part of the browser window.  If you do get a different window class for the area where the image is, then I'm sure there is a windows API that can get you a handle to it (possibly FindWindow would work).  Off the top of my head, I'd say the API would have a name something like EnumerateChildWindows and it would take the hWnd of the browser window -- the parent window -- as a parameter.... we can search for the right API once you've determined that you can get a window class name for that image area.

If you can't, then you could look at the HTML that google generates.  Sometimes, the position of an image on the screen is set to a fixed position.  If so, then you could capture the whole browser window into a bitmap, then you can use the BitBlt API to copy just a section of that bitmap to another bitmap, and that would be your image.

You could test this pretty easily by using your code above with the browsers window handle to capture the browser window to the clipboard, then paste it into Windows Paint and save the bitmap.  Then, do the same test two or three times, saving those browser screen captures.  Now, open those in Windows Paint and use the cursor to tell you where the start of the image is... if it is always in the same X,Y location, then you've got your starting point.  If the ending point is always in the same location, then you're pretty much done... just use BitBlt to copy that small rectangle from the original screen capture to another Bitmap and save that (or copy that to the clipboard).  I can help with that code if/when you're ready.

If the start point is always in the same area, but the end point is not, again, you might be able to parse the HTML.  Frequently, they will specify the height and width of the image to be displayed (sometimes they don't), that might help you to figure out the end point.

Often, HTML is coded to change the start point of objects on the screen based on the width of the browser window (they'll use Left or Top as a percentage).  This means that your image start point may show up at different X,Y locations depending on browsers window size.  That will be the hardest of all to deal with.  Then, your only chance will be that SPY can find a window class for that image section in the HTML.

References on the Windows API are usually not very helpful, and reading them is like reading your VCR's instruction manual.  The most helpful source I've found is Karl Peterson's website:

http://vb.mvps.org/

I download his samples and try them out in VB6 (he might have .NET examples now).  He uses a lot of Windows APIs and so his working samples really help.  In fact, maybe he has something close to what you're trying to do!

One thing you should be aware of, some of the Windows API signatures have changed under Windows XP and Vista, particularly those associated with Bitmaps (Can't remember which ones off the top of my head).  But, suffice it to say that your code might work on some computers but not others (that have XP or Vista installed)

Anyway, what you're attempting is very challenging, you're to be commended for getting this far!
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20300005
There is one other avenue you might want to explore.  Since you are working in Excel, Excel has something called a Web Query.  I don't have Excel on this machine, but if I remember correctly, there is a Data menu, then down near the bottom there should be an option for Query or Web Query, click that and choose New Query.

You'll get a textbox to enter your URL, so, type in the URL that will get you to the page with the image you're interested in.  When the wizard navigates to that page, move your cursor around and you'll see various parts of that page highlight.  I've used this to highlight table data on an HTML page and capture that data into an Excel spreadsheet... works pretty well.  But, it just might work for your image too.

If you're able to select your image.  Then, you can either save the Query as a query file to be able to run by choosing Run Query.  Or, you can save it as a Macro.  Saving it as a macro will give you the exact VBA code you need to find and select this image at will.

You will need to have installed the add-on called Microsoft Query Engine which is optional when you install Excel, so, if it is not installed, you'd need your Office disks to install it.
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20314567
Hi mdougan,

First of all, thanks for your valuable help and, quite honestly, without your insight and other
threads on EE I would be helplessly lost. You were right, this is a challenge!

I'm in a VBA Excel environment, not in VB.Net, and I haven't had any luck with the Spy utility.
When I Google "Spy", there is a great abundance of hits, but none seem to be what I'm looking for.

The code to find the IE Window I need works great, Once I've found it, I don't understand what the
IE Window is composed of.  In the IE window does each item or control (text boxes, buttons or images)  have a handle or how do I get info on each of these items and their content?

In looking at the Source for the IE HTML window, I came across this.

      cellLeft.innerHTML = '<form method="post" action="' + targetPage + '"><img       
      src="aspserialno.asp" alt="Serial Number Image" width="100" height="25" />

Can I examine each of the controls within the IE window and find the one I want by looking at its
width, heigh and "alt" content? If so, how do I expose each control so I can look for this
information.

As a secondary approach, as you suggested I'm looking in to Query tables to see what they have to
offer.

Thanks again for your direction and patience.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20315788
Spy, or more recently Spy++ was distributed with Visual Studio as one of the visual studio tools.  I may have the tool on one of my other machines.  Can you give me an example of a URL that I could navigate to, and then I could use the Spy tool to see if I can get a direct reference to that picture on the browser window.  I'm pretty sure that the elements in the window are addressable, but, Spy will probably tell me for sure.

If that little bit of HTML was the only HTML on the page, you could make the assumption that the upper left of the image was at 0,0 and the lower right was at 100,25, however, if there is any other HTML then this item will probably be offset by whatever else is defined in the page.

If you are using a fairly recent version of Excel, you could load your cellLeft.innerHTML into a DOM Document.  Unfortunately, all my examples are from .NET, but there should be many examples on the internet or here at EE.  DOM stands for Document Object Model and I've seen code where once the HTML is loaded into the DOM Document, you can directly address DIV and TABLE tags... so, I'm guessing that you could address other document objects as well.

Send me a sample URL and I'll see what I can see.

My pleasure to help, that's what this site is here for!

Mike
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20321882
Mike,

Thank you very much for offering to use SPY++ to help me make some progress.

The URL that you need to use SPY++ on is:

http://www.cookcountyassessor.com/filings/searchflat//search_res.asp

After you enter the URL, Then click on any of the blue numbers in the left hand column. After the click you the see what I trying to understand. I need to create a text string that has the blue numbers in the gray box. I've already created a 'Dictionary' of the pixel patterns for all the numeric characters and I need to be able to get the pixels from the screen image to my program. Then I can compare the pixel patterns to my dictionary to derive a character to use as part of my response to this prompt.

I'm working in a "read only" mode here, just trying to extract publically available information in an automated fashon. I not trying to change anything on the host web site.

If you ever run across a way for me to download SPY++ let me know. Tools like this are very handy when you come into a situation like this.

John
 
0
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20322484
There is FREE tool similar to Spy++ called WinSpector:
http://www.windows-spy.com/

(I have personally used it without any problems on my system)
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20323385
Hi John,

I'll give it a try right now and let you know what I find out.  It will be helpful for me to look at exactly what you are going after too.

You should go to the Karl Peterson site I listed above, and go to his download page.  He has a tool at the very bottom that he mentions works like SPY does... but then, it might be source code only.  If I can zip up the Spy tool, I might be able to put it somewhere that you could download it.
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20324287
Hi,

Well, good news and bad news.  I used Spy++ on that web page and it could only distinguish the web page window as a whole, not address any specific objects on the page.  

I looked at the source code for the page and noticed a little java script function that inserts a new row in the grid on the page to display the little security code image... that was part of the javascript code that you posted above referencing the innerHTML.  I noticed that they gave us the width and height of the security code image.  That got me thinking in another direction.

I did a screen print of the browser window and used a function called GetPixel to get the color value for the background color of the security code image.  It turns out that this color is only used by the background for the security code.  So, I put the bitmap into a picture box called Picture1 in a little sample VB program.  I also added a blank picture box Picture2 and a command button Command1.  Then, wrote this little routine.

Private Declare Function GetPixel Lib "gdi32" (ByVal hdc As Long, ByVal x As Long, ByVal y As Long) As Long
Private Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
Const SRCCOPY = &HCC0020

Private Sub Command1_Click()
Dim x As Long
Dim y As Long
For y = 0 To Picture1.Height
    For x = 0 To Picture1.Width
      lColor = GetPixel(Picture1.hdc, x, y)
      If lColor = 14540253 Then
         BitBlt Picture2.hdc, 0, 0, 86, 21, Picture1.hdc, x, y, SRCCOPY
         Exit Sub
      End If
    Next x
Next y
End Sub

What you can try, is to use the code that you already have to find the browser window and get the hDC for it.  Then, use the GetWindowRect to get the dimension for the browser window so that you can use that in place of where I have Picture1.Width and Picture1.Height  (width is the X dimension and Height is the Y dimension).  Then, create a new bitmap the way you have been doing, however, you can create it with a width of 86 and a  height of 21 pixels.  

Then BitBlt using the hDC of the memory bitmap, 0 dest X, 0 dest Y, 86 width, 21 height, the hDC of the browser window, the X and Y position of the first pixel with a color value of 14540253, SRCCOPY

Then, you can do whatever you want with that memory bitmap... copy it to the clipboard as you've been doing etc.

See if you can make any sense of that!
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20327812
Dang, close, but no cigar.

I tried this test, but the GetPixel doesn't seem to work on a bitmap that is in memory, versus a bitmap that is loaded into a picturebox control.  Hummm... you might be able to find an example somewhere where they've figured out how to do that.  I'll give you the code below as a starting point.  The approach has promise, and I know it can work in VB with physical controls, just this one little hump to get over to get it working in VBA.

I'm going to be out of town for the rest of the week with little time to check in, so, take your time!
Public Type RECT
        Left As Long
        Top As Long
        Right As Long
        Bottom As Long
End Type
 
Declare Function GetPixel Lib "gdi32" (ByVal hdc As Long, ByVal x As Long, ByVal y As Long) As Long
Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long
Declare Function GetWindowRect Lib "user32" (ByVal hwnd As Long, lpRect As RECT) As Long
Declare Function GetDC Lib "user32" (ByVal hwnd As Long) As Long
Declare Function CreateCompatibleBitmap Lib "gdi32" (ByVal hdc As Long, ByVal nWidth As Long, ByVal nHeight As Long) As Long
Declare Function CreateCompatibleDC Lib "gdi32" (ByVal hdc As Long) As Long
Declare Function SelectObject Lib "gdi32" (ByVal hdc As Long, ByVal hObject As Long) As Long
Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
Declare Function OpenClipboard Lib "user32" (ByVal hwnd As Long) As Long
Declare Function EmptyClipboard Lib "user32" () As Long
Declare Function SetClipboardData Lib "user32" (ByVal wFormat As Long, ByVal hMem As Long) As Long
Declare Function CloseClipboard Lib "user32" () As Long
Declare Function DeleteDC Lib "gdi32" (ByVal hdc As Long) As Long
Declare Function ReleaseDC Lib "user32" (ByVal hwnd As Long, ByVal hdc as Long) As Long
Public Const SRCCOPY = &HCC0020 ' (DWORD) dest = source
Public Const CF_BITMAP = 2
 
Sub ScreenToClipBoard()
Dim bHwnd As Long
Dim hdc As Long
Dim hdcMem As Long
Dim hBitmap As Long
 
Dim RECT As RECT
Dim junk As Long
Dim fwidth As Long
Dim fheight As Long
Dim x As Long
Dim y As Long
Dim lColor As Long
 
    '---------------------------------------------------
    ' Your code to open a browser window and navigate to 
    ' the site goes here
    '---------------------------------------------------
 
    '---------------------------------------------------
    ' Get window handle to Window
    '---------------------------------------------------
    ' I have Mozilla on that particular machine, use whatever works for you
    bHwnd = FindWindow("MozillaWindowClass", "Welcome to the Cook County Assessor's Virtual Office - Mozilla")
 
    '---------------------------------------------------
    ' Get the device context of the window and allocate memory
    '---------------------------------------------------
 
    hdc = GetDC(bHwnd)
    hdcMem = CreateCompatibleDC(hdc)
    hBitmap = CreateCompatibleBitmap(hdc, 86, 21)
    If hBitmap <> 0 Then
       junk = SelectObject(hdcMem, hBitmap)
    Else
       MsgBox "Couldn't create mem bitmap"
       Exit Sub
    End If
    
    '---------------------------------------------------
    ' Get screen dimensions of the browser window
    '---------------------------------------------------
    Call GetWindowRect(bHwnd, RECT)
    fwidth = RECT.Right - RECT.Left
    fheight = RECT.Bottom - RECT.Top
 
    '---------------------------------------------------
    ' Look at each pixel for the color behind the sec code
    '---------------------------------------------------
 
    For y = 0 To fheight
        For x = 0 To fwidth
            lColor = GetPixel(hdc, x, y)
            If lColor = 14540253 Then
                BitBlt hdcMem, 0, 0, 86, 21, hdc, x, y, SRCCOPY
                '---------------------------------------------
                ' Set up the Clipboard and copy bitmap
                '---------------------------------------------
                junk = OpenClipboard(bHwnd)
                junk = EmptyClipboard()
                junk = SetClipboardData(CF_BITMAP, hBitmap)
                junk = CloseClipboard()
                GoTo CleanUp
            End If
        Next x
    Next y
 
CleanUp:
    '---------------------------------------------
    ' Clean up handles
    '---------------------------------------------
    junk = DeleteDC(hdcMem)
    junk = ReleaseDC(bHwnd, hdc)
    Beep
    
 End Sub

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20330919
Mike,
Thanks for the second update. I was able to only spend a couple of hours on the code in your first suggestion and I didn't have much positive luck. I won't go into the details now since I just got your second suggestion.

I really appreciate you going "above and beyond" by taking the time to go to the web site and use SPY++ on it. I was able to get a copy of 'Winspector' from the site suggested by Idle Mind. It shows lots of interesting stuff and hopefully in time, I'll be able to understand what I'm looking at a little better.

I'll probably continue to work on this over the long weekend, but I hope that you enjoy the long Thanksgiving weekend.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20335437
Thanks John,

Hope you have some success with it.  I'm not sure if my approach was clear in the above code, but basically what I was trying to do was capture the entire browser window as a bitmap, then, search each pixel looking for a color that matches the background color of the security code area.  That should give you the upper left corner of that area.  Then, we already know the width and height, so, we could copy just that section of the bitmap to a new bitmap.  The problem is that the GetPixel function works well if the bitmap is loaded into a PictureBox control, however, it doesn't seem to work with the kind of bitmap that our screenshot was loaded into.

There might be some other object we could copy the screenshot into, such as a Standard Picture object, and perhaps GetPixel would work with that.  Things to consider if you're searching through code examples this weekend.

Mike
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20352793
Mike,

I understand your approach and it looks like a logical way to do what we want to do. However, when I test my code the only value returned from the GetPixel function is a -1. I think this means we're not looking at what we think we're looking at.

Unfortunately, I didn't get the time I had hoped for over the long weekend to work on this and it looks like I've got a few other fires that need my attention during the first day or so of this week.

I noticed a 'ScreentoClipBoard' function earlier and I'll try to do something with it when time is available. If you think of anything, please let me know.

A copy of my code at this point is included.

Hope you enjoyed your weekend.

John
    IEHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE
    If IEHwnd = 0 Then Stop  ' Did not find Window with the "WindowTitleText".
 
    ' Get the Device context of the window and allocate memory
    HDC = GetDC(IEHwnd)
    HDCmem = CreateCompatibleDC(HDC)
    hBitMap = CreateCompatibleBitmap(HDC, 86, 21)
    If hBitMap <> 0 Then
        junk = SelectObject(HDCmem, hBitMap)
        Else
        Stop ' Couldn't create mem bitmap
        Exit Sub
    End If
    
    ' Get screen dimensions of IE Window
    Call GetWindowRect(IEHwnd, Rect)          ' Get the borders of the IE Window.
    fwidth = Rect.Right - Rect.Left
    fheight = Rect.Bottom - Rect.Top
    
    success = False
    For y = 0 To fheight
        For x = 0 To fwidth
        lcolor = GetPixel(HDC, x, y)
        If lcolor <> -1 Then Stop
        If lcolor = 14540253 Then
            BitBlt HDCmem, 0, 0, 86, 21, HDC, x, y, SRCCOPY
            success = True
            Exit For
        End If
        Next x
    Next y
    
    If success = True Then Stop

Open in new window

0
 
LVL 18

Expert Comment

by:mdougan
ID: 20361022
By George, I think I've got it :-)

I found out that for GetPixel to work, you have to use the SelectObject on the hDC and a bitmap first.

So, I modified the code a little bit:

First, I get the browsers window handle using FindWindow

Next, I found it was necessary to bring the browser window to the foreground, otherwise overlapping window images will be included in the BitBlt.  Since it takes a second for a window to move to the foreground, I did a few DoEvents to give it enough time.

Get the dimensions of the source browser window into the Rect

Get the hDC for the browser window and create a compatible bitmap for it

Use SelectObject on the browser window/bitmap

Create a target hDC and bitmap

Search the source bitmap using GetPixel, looking for the backcolor of the security code

When found, select the target bitmap into the target hDC using SelectObject

BitBlt the source bitmap into the target

Copy the target bitmap into the Clipboard

In this example, I paste the clipboard into the first cell of the active spreadsheet

Important note, when cleaning up the handles, be sure to delete the bitmaps, otherwise you'll have a large memory leak!  You might want to search other examples and ensure that you are deleting/releasing all the necessary objects and handles, as a memory leak is bad news!

This code sample assumes that you've already created the browser window and navigated to the URL and expanded one of the rows to expose the security code.  Also make sure to substitute your FindWindow code for the code in the example that is only valid for my Mozilla browser.

Whew!
Public Type RECT
        Left As Long
        Top As Long
        Right As Long
        Bottom As Long
End Type
 
Declare Function GetPixel Lib "gdi32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long) As Long
Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long
Declare Function GetWindowRect Lib "user32" (ByVal hwnd As Long, lpRect As RECT) As Long
Declare Function GetDC Lib "user32" (ByVal hwnd As Long) As Long
Declare Function CreateCompatibleBitmap Lib "gdi32" (ByVal hDC As Long, ByVal nWidth As Long, ByVal nHeight As Long) As Long
Declare Function CreateCompatibleDC Lib "gdi32" (ByVal hDC As Long) As Long
Declare Function SelectObject Lib "gdi32" (ByVal hDC As Long, ByVal hObject As Long) As Long
Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
Declare Function OpenClipboard Lib "user32" (ByVal hwnd As Long) As Long
Declare Function EmptyClipboard Lib "user32" () As Long
Declare Function SetClipboardData Lib "user32" (ByVal wFormat As Long, ByVal hMem As Long) As Long
Declare Function CloseClipboard Lib "user32" () As Long
Declare Function DeleteDC Lib "gdi32" (ByVal hDC As Long) As Long
Declare Function ReleaseDC Lib "user32" (ByVal hwnd As Long, ByVal hDC As Long) As Long
Declare Function DeleteObject Lib "gdi32" (ByVal hObject As Long) As Long
 
Declare Function SetForegroundWindow Lib "user32" (ByVal hwnd As Long) As Long
 
Public Const SRCCOPY = &HCC0020 ' (DWORD) dest = source
Public Const CF_BITMAP = 2
 
 
Sub ScreenToClipBoard()
Dim bHwnd As Long
Dim hDC As Long
Dim hBitmap As Long
Dim hdcMem2 As Long
Dim hBitmap2 As Long
 
Dim RECT As RECT
Dim junk As Long
Dim fwidth As Long
Dim fheight As Long
Dim x As Long
Dim y As Long
Dim lColor As Long
 
    '---------------------------------------------------
    ' Insert your code here to create the browser window
    ' and navigate to your URL
    '---------------------------------------------------
 
    '---------------------------------------------------
    ' Get window handle to Browser
    ' change this as necessary to work with your browser
    '---------------------------------------------------
    bHwnd = FindWindow("MozillaWindowClass", "Welcome to the Cook County Assessor's Virtual Office - Mozilla")
    
    ' The browser window has to be on top of all other windows, otherwise their images will overlay the browser window's
    junk = SetForegroundWindow(bHwnd)
    
    ' There will be a lag before the window is actually in the foreground, so, give it a little CPU to move to the front
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    
    '---------------------------------------------------
    ' Get screen coordinates of the browser window
    '---------------------------------------------------
    Call GetWindowRect(bHwnd, RECT)
    fwidth = RECT.Right - RECT.Left
    fheight = RECT.Bottom - RECT.Top
        
    '---------------------------------------------------
    ' Get a handle to the browser window
    '---------------------------------------------------
    hDC = GetDC(bHwnd)
    
    '---------------------------------------------------
    ' GetPixel only works if the hDC and bitmap have been selected with SelectObject
    ' So, create a compatible bitmap and select it
    '---------------------------------------------------
    hBitmap = CreateCompatibleBitmap(hDC, fwidth, fheight)
    If hBitmap <> 0 Then
       junk = SelectObject(hDC, hBitmap)
    Else
       MsgBox "Couldn't create mem bitmap"
       Exit Sub
    End If
 
    '---------------------------------------------------
    ' Now, create an hDC and Bitmap for the final result bitmap
    '---------------------------------------------------
    hdcMem2 = CreateCompatibleDC(hDC)
    hBitmap2 = CreateCompatibleBitmap(hDC, 86, 21)
            
    '---------------------------------------------------
    ' Search the browser window looking for the backcolor
    ' of the security code image
    '---------------------------------------------------
    For x = 0 To fheight - 1
        For y = 0 To fwidth - 1
            lColor = GetPixel(hDC, x, y)
            If lColor = 14540253 Then
                '---------------------------------------------
                ' Once the security code is found, prepare to copy it
                ' to another memory resident bitmap
                '---------------------------------------------
                
                If hBitmap2 <> 0 Then
                  junk = SelectObject(hdcMem2, hBitmap2)
                Else
                  MsgBox "Couldn't create mem bitmap"
                  Exit Sub
                End If
 
                BitBlt hdcMem2, 0, 0, 86, 21, hDC, x, y, SRCCOPY
                
                '---------------------------------------------
                ' Set up the Clipboard and copy memory resident bitmap to it
                '---------------------------------------------
                junk = OpenClipboard(bHwnd)
                junk = EmptyClipboard()
                junk = SetClipboardData(CF_BITMAP, hBitmap2)
                junk = CloseClipboard()
                
                '---------------------------------------------
                ' Optionally, paste the clipboard contents into a cell on the spreadsheet
                '---------------------------------------------
                Range("A1").Select
                ActiveSheet.Paste
                
                GoTo CleanUp
            End If
        Next y
    Next x
 
CleanUp:
    '---------------------------------------------
    ' Clean up handles, important, don't forget to
    ' delete the bitmaps or you will have a memory leak!
    '---------------------------------------------
    junk = DeleteObject(hBitmap)
    junk = DeleteObject(hBitmap2)
    junk = DeleteDC(hdcMem2)
    junk = ReleaseDC(bHwnd, hDC)
    
 End Sub
 

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20365790
Mike,

Your last response made my day !! , Yes, it can be done !!!. I changed my code and unfortunately, it still doesn't work for me. I added a few lines of debugging code to make sure that when I try to use the 'SelectObject' that it returns a non zero value. It doesn't, I always get zero.

Do we possibly have a problem that I am using IE and you are using Mozilla or is it something else that I just can't find in my code? I would appreciate you looking at the snippet to see if I'm missing something somewhere. If it is an incompatibility between IE/Mozilla, I'm certainly not adverse to using Mozilla if that what it takes to make it work. However, since I never used it before I want to proceede with caution to make sure nothing else is adversely affected.

Thanks again for your support and patience. If you're ever in Chicago, the drinks are on me.

John


 
Private Sub ScreenToClipBoard()
Dim bHwnd As Long
Dim fwidth As Long
Dim fheight As Long
Dim hBitMap As Long
Dim hBitMap2 As Long
Dim hDC As Long
Dim hDCmem2 As Long
Dim lColor As Long
Dim junk As Long
Dim Rect As Rect
Dim x As Long
Dim y As Long
 
 
Stop
    bHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE
    If bHwnd = 0 Then
        MsgBox ("Did not find the Window with the title of:" & vbCrLf & vbCrLf & _
                """" & WindowTitleText & """" & vbCrLf & vbCrLf & _
                "Error number=" & Err.LastDllError)
        Exit Sub
    End If
 
 
    junk = SetForegroundWindow(bHwnd)   ' Set this handle/Window as the Foreground Window.
    If junk = 0 Then                    ' A zero return code indicates failure
        MsgBox ("Unable to Set IE as the foreground Window.")
        Exit Sub
    End If
    
    '---------------------------------------------------
    ' There will be some lag before this window is actually in the foreground, so, give it a
    ' little CPU to move it to the front
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    
    '---------------------------------------------------
    ' Get screen coordinates of the browser window
    '---------------------------------------------------
    Call GetWindowRect(bHwnd, Rect)
    fwidth = Rect.Right - Rect.Left         ' Calculate Width of Window
    fheight = Rect.Bottom - Rect.Top        ' Calculate Height of Window
    
    '---------------------------------------------------
    ' Get a handle to the browser window
    '---------------------------------------------------
    hDC = GetDC(bHwnd)                      ' Get 'Device Context' for IE window.
    
    '---------------------------------------------------
    ' GetPixel only works if the hDC and the bitmap have been selected with SelectObject
    ' So, create a compatible bitmap and select it.
    '---------------------------------------------------
    hBitMap = CreateCompatibleBitmap(hDC, fwidth, fheight)
    If hBitMap <> 0 Then
        junk = SelectObject(hDC, hBitMap)
        If junk = 0 Then                        ' In my testing I always get a zero
            MsgBox ("Unable to SelectObject")   ' See http://msdn2.microsoft.com/en-us/library/ms533272.aspx
            Exit Sub
        End If
    Else
        MsgBox "Could NOT create a memory bitmap"
        Exit Sub
    End If
    
    '----------------------------------------------------
    ' Now create a hDC and bitmap for the final result bitmap.
    '----------------------------------------------------
    hDCmem2 = CreateCompatibleDC(hDC)
    hBitMap2 = CreateCompatibleBitmap(hDC, 86, 21)
    
    '----------------------------------------------------
    ' Search the browser window looking for the backcolor
    ' of the security code image
    '----------------------------------------------------
    For x = 0 To fheight - 1
        For y = 0 To fwidth - 1
            lColor = GetPixel(hDC, x, y)
            If lColor <> -1 Then Stop       ' My testing never stop here
            If lColor = 14540253 Then
                '---------------------------------------------
                ' Once the security code is found, prepare to copy it
                ' to another memory resident bitmap
                '---------------------------------------------
                
                If hBitMap2 <> 0 Then
                    junk = SelectObject(hDCmem2, hBitMap2)
                Else
                    MsgBox ("Could NOT create memory bitmap")
                    Exit Sub
                End If
            
                BitBlt hDCmem2, 0, 0, 86, 21, hDC, x, y, SRCCOPY
                
                '---------------------------------------------
                ' Set up the Clipboard and copy memory resident bitmap to it
                '---------------------------------------------
                junk = OpenClipboard(bHwnd)
                junk = EmptyClipboard()
                junk = SetClipboardData(CF_BITMAP, hBitMap2)
                junk = CloseClipboard()
        
                '---------------------------------------------
                ' Optionally, paste the clipboard contents into a cell on the spreadsheet
                '---------------------------------------------
                Range("A1").Select
                ActiveSheet.Paste
                
                '---------------------------------------------
                ' Clean up handles
                '---------------------------------------------
                junk = DeleteObject(hBitMap)
                junk = DeleteObject(hBitMap2)
                junk = DeleteDC(hDCmem2)
                junk = ReleaseDC(bHwnd, hDC)
            End If
        Next y
    Next x
 
Stop
 
End Sub

Open in new window

0
 
LVL 18

Expert Comment

by:mdougan
ID: 20367278
Hi,

No, I don't think it would have anything to do with which browser we're using... the only thing that should be affected by that is the FindWindow, and as long as you are getting back a valid window handle from that call, you should be OK.  

More likely is that there may be a difference in our operating systems.  The computer that I have Excel on is pretty old, and I'm running Windows NT on that machine.  I know that some of these low level graphics routines changed under XP, so, it is possible that things which work under NT will not work under XP.

If GetPixel is always returning a -1, then that means that the hDC is not properly selected.

I went to the link you provided in the code and noticed a statement that said that bitmaps can only be selected for memory DCs.  I'd tried to be efficient and just use the hDC of the physical browser window, since we have it.  However, that may not be considered a memory DC.  So, I've taken your code and I've created another compatible hDCmem, selected the hBitMap into the hDCmem and then did a BitBlt of the browser window into that hDCmem.  Then, everywhere below that line of code, where hDC was referenced, I changed the code to reference the hDCmem.  I wasn't sure if the BitBlt should come before or after the SelectObject, I guessed after, but if it doesn't seem to work, then you might try it before.

I'm not on the computer that has Excel, so, I can't test it, but give it a try and let me know how it goes!

I'll be happy to take you up on your offer if I get out to Chicago sometime.  I lived in Chicago for a while, a few years back.  It's a great place.  But I'm in NY now.
Private Sub ScreenToClipBoard()
Dim bHwnd As Long
Dim fwidth As Long
Dim fheight As Long
Dim hBitMap As Long
Dim hBitMap2 As Long
Dim hDC As Long
Dim hDCmem As Long
Dim hDCmem2 As Long
Dim lColor As Long
Dim junk As Long
Dim Rect As Rect
Dim x As Long
Dim y As Long
 
 
    bHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE
    If bHwnd = 0 Then
        MsgBox ("Did not find the Window with the title of:" & vbCrLf & vbCrLf & _
                """" & WindowTitleText & """" & vbCrLf & vbCrLf & _
                "Error number=" & Err.LastDllError)
        Exit Sub
    End If
 
 
    junk = SetForegroundWindow(bHwnd)   ' Set this handle/Window as the Foreground Window.
    If junk = 0 Then                    ' A zero return code indicates failure
        MsgBox ("Unable to Set IE as the foreground Window.")
        Exit Sub
    End If
    
    '---------------------------------------------------
    ' There will be some lag before this window is actually in the foreground, so, give it a
    ' little CPU to move it to the front
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    DoEvents
    
    '---------------------------------------------------
    ' Get screen coordinates of the browser window
    '---------------------------------------------------
    Call GetWindowRect(bHwnd, Rect)
    fwidth = Rect.Right - Rect.Left         ' Calculate Width of Window
    fheight = Rect.Bottom - Rect.Top        ' Calculate Height of Window
    
    '---------------------------------------------------
    ' Get a handle to the browser window
    '---------------------------------------------------
    hDC = GetDC(bHwnd)                      ' Get 'Device Context' for IE window.
    
    '---------------------------------------------------
    ' GetPixel only works if the hDC and the bitmap have been selected with SelectObject
    ' So, create a compatible bitmap and select it.
    '---------------------------------------------------
    hDCmem = CreateCompatibleDC(hDC)
    hBitMap = CreateCompatibleBitmap(hDC, fwidth, fheight)
 
    ' Not sure if this should be before or after the SelectObject
    'BitBlt hDCmem, 0, 0, fwidth, fheight, hDC, 0, 0, SRCCOPY
 
 
    If hBitMap <> 0 Then
        junk = SelectObject(hDCmem, hBitMap)
        If junk = 0 Then                        ' In my testing I always get a zero
            MsgBox ("Unable to SelectObject")   ' See http://msdn2.microsoft.com/en-us/library/ms533272.aspx
            Exit Sub
        End If
    Else
        MsgBox "Could NOT create a memory bitmap"
        Exit Sub
    End If
   
    ' Not sure if this should be before or after the SelectObject
    BitBlt hDCmem, 0, 0, fwidth, fheight, hDC, 0, 0, SRCCOPY
 
 
    '----------------------------------------------------
    ' Now create a hDC and bitmap for the final result bitmap.
    '----------------------------------------------------
    hDCmem2 = CreateCompatibleDC(hDC)
    hBitMap2 = CreateCompatibleBitmap(hDC, 86, 21)
    
    '----------------------------------------------------
    ' Search the browser window looking for the backcolor
    ' of the security code image
    '----------------------------------------------------
    For x = 0 To fheight - 1
        For y = 0 To fwidth - 1
            lColor = GetPixel(hDCmem, x, y)
            If lColor = 14540253 Then
                '---------------------------------------------
                ' Once the security code is found, prepare to copy it
                ' to another memory resident bitmap
                '---------------------------------------------
                
                If hBitMap2 <> 0 Then
                    junk = SelectObject(hDCmem2, hBitMap2)
                Else
                    MsgBox ("Could NOT create memory bitmap")
                    Exit Sub
                End If
            
                BitBlt hDCmem2, 0, 0, 86, 21, hDCmem, x, y, SRCCOPY
                
                '---------------------------------------------
                ' Set up the Clipboard and copy memory resident bitmap to it
                '---------------------------------------------
                junk = OpenClipboard(bHwnd)
                junk = EmptyClipboard()
                junk = SetClipboardData(CF_BITMAP, hBitMap2)
                junk = CloseClipboard()
        
                '---------------------------------------------
                ' Optionally, paste the clipboard contents into a cell on the spreadsheet
                '---------------------------------------------
                Range("A1").Select
                ActiveSheet.Paste
                
                '---------------------------------------------
                ' Clean up handles
                '---------------------------------------------
                junk = DeleteObject(hBitMap)
                junk = DeleteObject(hBitMap2)
                junk = DeleteDC(hDCmem)
                junk = DeleteDC(hDCmem2)
                junk = ReleaseDC(bHwnd, hDC)
            End If
        Next y
    Next x
 
Stop
 
End Sub

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20368751
Mike,
Wow, it works great!!! I still have to do some more code to make sure I handle the bitmaps correctly but ALL the points will go to you. I am extremely grateful for your help, support and encouragement on this. I thought it could be done this way, and with your expertise you made it happen.

I'll post here again if I run into other problems related to this and also to notify you when I sure everything works the way it should.

Thanks again,

John
0
 
LVL 18

Accepted Solution

by:
mdougan earned 500 total points
ID: 20373516
My pleasure. I enjoy a challenge and this was a good one!

Some people just want you to hand them a completed solution, but I prefer working "with" the person asking the question.  You are great that way.

Just comment here if you open up another item you want me to look at and I'll be notified.  And close this item whenever you are satisfied with it.

Cheers!
0
 
LVL 1

Author Closing Comment

by:PropertyTaxAnalysts
ID: 31409174
This was a difficult problem and mdougan did an excellent job in helping me develop a solution and more importantly, he took the time and had enough patience to make sure I understood the solution also.

He's the kind of guy you want to be sure you keep as a resource!
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Problem to error 4 67
Problem to With line 4 63
Event ID 1054 Userenv 2 33
convert Systemjs to Webpack 3 30
I was working on a PowerPoint add-in the other day and a client asked me "can you implement a feature which processes a chart when it's pasted into a slide from another deck?". It got me wondering how to hook into built-in ribbon events in Office.
When you start your Windows 10 PC and got an "Operating system not found" error or just saw  "Auto repair for startup" or a blinking cursor with black screen. A loop for Auto repair will start but fix nothing.  You will be panic as there are no back…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question