Solved

Capture Screen Content or Window Content

Posted on 2007-11-14
24
3,969 Views
Last Modified: 2012-05-05
Using the examples I found earlier on EE I am able to capture the content of the screen. How ever, the only thing I see is a copy of my VBA source code, and not the "Other" window that I need to examine.

I need some help in understand how this part of windows works.

  1. Is there a 'Collection' of windows somewhere and if so, how do I reference it and get the count of windows.

  2. How do I identify each of the windows in the collection to find the one I need? Is there text or something else that I can examine to determine if I found the window I need.
 
 3. After finding the desired window, is there a map or directory that has specifics info. on the client controls (frames, textboxes, images, etc) that appear on the screen.

4. How can I get the content of these controls to variables in my VBA code. Specifically the content of an .bmp and .jpg image.
 
0
Comment
Question by:PropertyTaxAnalysts
  • 11
  • 10
  • 3
24 Comments
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20280814
It would help if you would show us the code you are currently attempting to use...

Also, is the "Other" window part of YOUR app?...or an EXTERNAL app?

The answers to your questions are dependent upon the environment you are working in.  If you are using VB6 and the window is part of your app then the things in question are relatively easy.  VBA doesn't keep a Forms collection as far as I know though.

If you are talking about external windows then it becomes fairly complicated as you must Windows APIs to enumerate all the windows.  There isn't a direct method to get a count.  Windows can be indentifed by their caption and/or ClassName.  It is possible to enumerate over the controls in a specific window and determine their classname and/or text.

So please give us more details about your environment and specifically what you are tyring to do...
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20281786
Idle Mind,

Thanks for your quick response. I appoligize for leaving out so me prerequisite details. I'm working in Excel VBA and I have programitically started Internet Explorer, navigated to the correct web site, and waited for the server to populate the IE window.  From my VBA code I need to find the IE window and extract a .bmp image from the screen.

The testing that I've done (sample below) so far only shows a window of my executing code. It looks like it only handles my task, and nothing else that may be in another window on the screen.  

I don't need a count as long as I can find the IE window that I need.

Sample Code
'=============================================================================
Function ScreenToClipBoard()
Dim ActiveHwnd As Long
Dim DeskHwnd As Long
Dim ForegroundHwnd As Long
Dim hdc As Long
Dim hdcMem As Long
Dim rect As RECT_Type
Dim junk As Long
Dim fwidth As Long, fheight As Long
Dim hBitmap As Long
 
    '---------------------------------------------------
    ' Get window handle to Windows and Microsoft Access
    '---------------------------------------------------
    DeskHwnd = GetDesktopWindow()
    ActiveHwnd = GetActiveWindow()
    ForegroundHwnd = GetForegroundWindow()

    '---------------------------------------------------
    ' Get screen coordinates of Active Window
    '---------------------------------------------------
    Call GetWindowRect(ActiveHwnd, rect)
    ' or
    ' Call GetWindowRect(ForegroundHwnd, rect)
    fwidth = rect.right - rect.left
    fheight = rect.bottom - rect.top
 
    '---------------------------------------------------
    ' Get the device context of Desktop and allocate memory
    '---------------------------------------------------
    hdc = GetDC(DeskHwnd)
    hdcMem = CreateCompatibleDC(hdc)
    hBitmap = CreateCompatibleBitmap(hdc, fwidth, fheight)
 
    If hBitmap <> 0 Then
       junk = SelectObject(hdcMem, hBitmap)
 
       '---------------------------------------------
       ' Copy the Desktop bitmap to memory location
       ' based on Microsoft Access coordinates.
       '---------------------------------------------
       junk = BitBlt(hdcMem, 0, 0, fwidth, fheight, hdc, rect.left, _
                     rect.top, SRCCOPY)
 
       '---------------------------------------------
       ' Set up the Clipboard and copy bitmap
       '---------------------------------------------
       junk = OpenClipboard(DeskHwnd)
       junk = EmptyClipboard()
       junk = SetClipboardData(CF_BITMAP, hBitmap)
       junk = CloseClipboard()
    End If
 
    '---------------------------------------------
    ' Clean up handles
    '---------------------------------------------
    junk = DeleteDC(hdcMem)
    junk = ReleaseDC(DeskHwnd, hdc)
 
End Function
'=============================================================================
I hope that I've answered your questions, but if I still have omitted something you need, I'll try to respond quickly.

John
0
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20281824
Ok...so you're using the ScreenToClipBoard() method...but how are you calling it?

The code is grabbing the coordinates of the Active window:

    Call GetWindowRect(ActiveHwnd, rect)

So make sure you are not stealing focus before callling the code...

Also, how are you starting Internet Explorer?  Depending on the method, we may be able to get a handle to that window directly...
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20285869
Idle_Mind:

Below is the code I use from within my VBA program to bring up the Inernet Explorer. It also came from one of the EE contributors.

'=====================================================================
    Set IE = CreateObject("InternetExplorer.Application")
    Set objshell = CreateObject("WScript.Shell")

    With IE
        .left = 20
        .top = 20
        .Height = 500
        .Width = 950
        .MenuBar = True         ' Must be on since the 'Refresh' key may be used
        .Toolbar = True
        .StatusBar = True
        .navigate "http://www.google.com"
        .Visible = True
    End With
'================================================================

I'm using this because it sure seems to work, but if there is a better way, I'm all ears. I do have a need to extract and set HTML values (mainly strings) in the IE HTML from within my VBA code. So far that works also.  The .bmp and . jpg images I need are in popups and I haven't been able to find the variable name for these images within the HTML code. Please remember that I'm a real novice at the VBA programming and I barely qualify as a beginner for the HTML, but I'm eager to learn more about both of these areas.

I think I understand your comment about stealing the focus. When my VBA code executes the "IE.Navigate" command, IE becomes the active window, and when IE has completed my request, I become the active window again. If this is true, my code should only be able to capture the active window of itself. What can I do to circumvent this situation so I do not steal the focus from IE.

I hope this answers you questions.

John


   
       
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20291169
Hi John,

It does sound like you are having difficulty finding the correct window.  I don't think you can rely on either the Active Window or the Foreground Window.  What you could use is the windows API called FindWindow  to find the window you want.  You can either pass the exact windows title for that window, or, you can pass the "window class" to FindWindow to get the window handle.  If the window you spawn with the call to google always has the same title, you can use that.  

Otherwise, to get the window class of a window, you can use a tool such as SPY.  You probably have SPY listed as a tool under your Visual Studio Tools menu, but if not, google it and you can probaby download it free.  When you run SPY, you can move the cursor over any window on your screen and then click the window you're interested in and it will tell you the window class.  You can then use that in the FindWindow call to get a handle to that particular type of window.  Now, it might not work very well if you happen to have two or more IE browsers open on the screen at the same time as both will have the same window class.

Another thing you can check is to see if your object variable, IE has a property called hwnd, some of the VB/VBA object models include the window handle as a property.

Finally, once you navigate to the HTML page in your VBA code as above, the IE object should have some property that contains the HTML that was returned (might be InnerHTML, or Document, or Body -- don't really remember).  It would be possible for you to parse the HTML and find the reference in the HTML to the image's source.  The HTML would look something like:

<img src="http://www.bla.com/somepic.jpg"

I believe that you can then use LoadPicture to load the picture from its location into a picturebox or bitmap variable in your program.  That way, you're not relying on grabbing it off the screen, but rather getting it from its source.  If LoadPicture doesn't work, then maybe using a file stream object would.  You might search EE for Loading Bitmap from Internet or something like that.  I know I've participated in those threads before.

Good luck!
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20298885
mdougan,

I followed your suggestion and after the call to "FindWindow" the value of x is not zero.

Is the value of x the Hwnd value?

If so, I need to get the bitmap from the screen since its source seems to change the value every time I reference it. It's somewhat like a non repeatable serial number and the value on the screen is what I need.

Is there a map or directory that has specifics info. on the content of this window (frames, textboxes, images, etc), or do I need to develop some sort of "Pattern Matching" code that will enable me to find an anchor point of the .bmp image (x,y value of the top left corner).

After I find this point, what's the best way to examine the contents between the upper left anchor to the bottom right anchor so I can determine its content. Fortunately, the .bmp I need seems to always be in a rectangular shape.

Any suggested reading references for the Hwnd or DC controls/methods would be greatly appreciated since my brain does not yet have a firm concept/understanding of the tools/functions that I need to use to accomplish this objective.

Thanks for your suggestion and I feel that I've moved ahead by a Giant step.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20299906
Yes, the result of the FindWindow call is the window handle  (hWnd).

What you are trying to do is a very tricky thing.  It is "possible" that any text entry area, or image display area might have a name or handle that you can use to capture its contents.  Your best bet is to use that tool SPY, mouse over the image and/or click on the image in the browser and see if you get different values than when you mouse-over/click on any other part of the browser window.  If you do get a different window class for the area where the image is, then I'm sure there is a windows API that can get you a handle to it (possibly FindWindow would work).  Off the top of my head, I'd say the API would have a name something like EnumerateChildWindows and it would take the hWnd of the browser window -- the parent window -- as a parameter.... we can search for the right API once you've determined that you can get a window class name for that image area.

If you can't, then you could look at the HTML that google generates.  Sometimes, the position of an image on the screen is set to a fixed position.  If so, then you could capture the whole browser window into a bitmap, then you can use the BitBlt API to copy just a section of that bitmap to another bitmap, and that would be your image.

You could test this pretty easily by using your code above with the browsers window handle to capture the browser window to the clipboard, then paste it into Windows Paint and save the bitmap.  Then, do the same test two or three times, saving those browser screen captures.  Now, open those in Windows Paint and use the cursor to tell you where the start of the image is... if it is always in the same X,Y location, then you've got your starting point.  If the ending point is always in the same location, then you're pretty much done... just use BitBlt to copy that small rectangle from the original screen capture to another Bitmap and save that (or copy that to the clipboard).  I can help with that code if/when you're ready.

If the start point is always in the same area, but the end point is not, again, you might be able to parse the HTML.  Frequently, they will specify the height and width of the image to be displayed (sometimes they don't), that might help you to figure out the end point.

Often, HTML is coded to change the start point of objects on the screen based on the width of the browser window (they'll use Left or Top as a percentage).  This means that your image start point may show up at different X,Y locations depending on browsers window size.  That will be the hardest of all to deal with.  Then, your only chance will be that SPY can find a window class for that image section in the HTML.

References on the Windows API are usually not very helpful, and reading them is like reading your VCR's instruction manual.  The most helpful source I've found is Karl Peterson's website:

http://vb.mvps.org/

I download his samples and try them out in VB6 (he might have .NET examples now).  He uses a lot of Windows APIs and so his working samples really help.  In fact, maybe he has something close to what you're trying to do!

One thing you should be aware of, some of the Windows API signatures have changed under Windows XP and Vista, particularly those associated with Bitmaps (Can't remember which ones off the top of my head).  But, suffice it to say that your code might work on some computers but not others (that have XP or Vista installed)

Anyway, what you're attempting is very challenging, you're to be commended for getting this far!
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20300005
There is one other avenue you might want to explore.  Since you are working in Excel, Excel has something called a Web Query.  I don't have Excel on this machine, but if I remember correctly, there is a Data menu, then down near the bottom there should be an option for Query or Web Query, click that and choose New Query.

You'll get a textbox to enter your URL, so, type in the URL that will get you to the page with the image you're interested in.  When the wizard navigates to that page, move your cursor around and you'll see various parts of that page highlight.  I've used this to highlight table data on an HTML page and capture that data into an Excel spreadsheet... works pretty well.  But, it just might work for your image too.

If you're able to select your image.  Then, you can either save the Query as a query file to be able to run by choosing Run Query.  Or, you can save it as a Macro.  Saving it as a macro will give you the exact VBA code you need to find and select this image at will.

You will need to have installed the add-on called Microsoft Query Engine which is optional when you install Excel, so, if it is not installed, you'd need your Office disks to install it.
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20314567
Hi mdougan,

First of all, thanks for your valuable help and, quite honestly, without your insight and other
threads on EE I would be helplessly lost. You were right, this is a challenge!

I'm in a VBA Excel environment, not in VB.Net, and I haven't had any luck with the Spy utility.
When I Google "Spy", there is a great abundance of hits, but none seem to be what I'm looking for.

The code to find the IE Window I need works great, Once I've found it, I don't understand what the
IE Window is composed of.  In the IE window does each item or control (text boxes, buttons or images)  have a handle or how do I get info on each of these items and their content?

In looking at the Source for the IE HTML window, I came across this.

      cellLeft.innerHTML = '<form method="post" action="' + targetPage + '"><img       
      src="aspserialno.asp" alt="Serial Number Image" width="100" height="25" />

Can I examine each of the controls within the IE window and find the one I want by looking at its
width, heigh and "alt" content? If so, how do I expose each control so I can look for this
information.

As a secondary approach, as you suggested I'm looking in to Query tables to see what they have to
offer.

Thanks again for your direction and patience.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20315788
Spy, or more recently Spy++ was distributed with Visual Studio as one of the visual studio tools.  I may have the tool on one of my other machines.  Can you give me an example of a URL that I could navigate to, and then I could use the Spy tool to see if I can get a direct reference to that picture on the browser window.  I'm pretty sure that the elements in the window are addressable, but, Spy will probably tell me for sure.

If that little bit of HTML was the only HTML on the page, you could make the assumption that the upper left of the image was at 0,0 and the lower right was at 100,25, however, if there is any other HTML then this item will probably be offset by whatever else is defined in the page.

If you are using a fairly recent version of Excel, you could load your cellLeft.innerHTML into a DOM Document.  Unfortunately, all my examples are from .NET, but there should be many examples on the internet or here at EE.  DOM stands for Document Object Model and I've seen code where once the HTML is loaded into the DOM Document, you can directly address DIV and TABLE tags... so, I'm guessing that you could address other document objects as well.

Send me a sample URL and I'll see what I can see.

My pleasure to help, that's what this site is here for!

Mike
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20321882
Mike,

Thank you very much for offering to use SPY++ to help me make some progress.

The URL that you need to use SPY++ on is:

http://www.cookcountyassessor.com/filings/searchflat//search_res.asp

After you enter the URL, Then click on any of the blue numbers in the left hand column. After the click you the see what I trying to understand. I need to create a text string that has the blue numbers in the gray box. I've already created a 'Dictionary' of the pixel patterns for all the numeric characters and I need to be able to get the pixels from the screen image to my program. Then I can compare the pixel patterns to my dictionary to derive a character to use as part of my response to this prompt.

I'm working in a "read only" mode here, just trying to extract publically available information in an automated fashon. I not trying to change anything on the host web site.

If you ever run across a way for me to download SPY++ let me know. Tools like this are very handy when you come into a situation like this.

John
 
0
 
LVL 85

Expert Comment

by:Mike Tomlinson
ID: 20322484
There is FREE tool similar to Spy++ called WinSpector:
http://www.windows-spy.com/

(I have personally used it without any problems on my system)
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 18

Expert Comment

by:mdougan
ID: 20323385
Hi John,

I'll give it a try right now and let you know what I find out.  It will be helpful for me to look at exactly what you are going after too.

You should go to the Karl Peterson site I listed above, and go to his download page.  He has a tool at the very bottom that he mentions works like SPY does... but then, it might be source code only.  If I can zip up the Spy tool, I might be able to put it somewhere that you could download it.
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20324287
Hi,

Well, good news and bad news.  I used Spy++ on that web page and it could only distinguish the web page window as a whole, not address any specific objects on the page.  

I looked at the source code for the page and noticed a little java script function that inserts a new row in the grid on the page to display the little security code image... that was part of the javascript code that you posted above referencing the innerHTML.  I noticed that they gave us the width and height of the security code image.  That got me thinking in another direction.

I did a screen print of the browser window and used a function called GetPixel to get the color value for the background color of the security code image.  It turns out that this color is only used by the background for the security code.  So, I put the bitmap into a picture box called Picture1 in a little sample VB program.  I also added a blank picture box Picture2 and a command button Command1.  Then, wrote this little routine.

Private Declare Function GetPixel Lib "gdi32" (ByVal hdc As Long, ByVal x As Long, ByVal y As Long) As Long
Private Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long
Const SRCCOPY = &HCC0020

Private Sub Command1_Click()
Dim x As Long
Dim y As Long
For y = 0 To Picture1.Height
    For x = 0 To Picture1.Width
      lColor = GetPixel(Picture1.hdc, x, y)
      If lColor = 14540253 Then
         BitBlt Picture2.hdc, 0, 0, 86, 21, Picture1.hdc, x, y, SRCCOPY
         Exit Sub
      End If
    Next x
Next y
End Sub

What you can try, is to use the code that you already have to find the browser window and get the hDC for it.  Then, use the GetWindowRect to get the dimension for the browser window so that you can use that in place of where I have Picture1.Width and Picture1.Height  (width is the X dimension and Height is the Y dimension).  Then, create a new bitmap the way you have been doing, however, you can create it with a width of 86 and a  height of 21 pixels.  

Then BitBlt using the hDC of the memory bitmap, 0 dest X, 0 dest Y, 86 width, 21 height, the hDC of the browser window, the X and Y position of the first pixel with a color value of 14540253, SRCCOPY

Then, you can do whatever you want with that memory bitmap... copy it to the clipboard as you've been doing etc.

See if you can make any sense of that!
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20327812
Dang, close, but no cigar.

I tried this test, but the GetPixel doesn't seem to work on a bitmap that is in memory, versus a bitmap that is loaded into a picturebox control.  Hummm... you might be able to find an example somewhere where they've figured out how to do that.  I'll give you the code below as a starting point.  The approach has promise, and I know it can work in VB with physical controls, just this one little hump to get over to get it working in VBA.

I'm going to be out of town for the rest of the week with little time to check in, so, take your time!
Public Type RECT

        Left As Long

        Top As Long

        Right As Long

        Bottom As Long

End Type
 

Declare Function GetPixel Lib "gdi32" (ByVal hdc As Long, ByVal x As Long, ByVal y As Long) As Long

Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long

Declare Function GetWindowRect Lib "user32" (ByVal hwnd As Long, lpRect As RECT) As Long

Declare Function GetDC Lib "user32" (ByVal hwnd As Long) As Long

Declare Function CreateCompatibleBitmap Lib "gdi32" (ByVal hdc As Long, ByVal nWidth As Long, ByVal nHeight As Long) As Long

Declare Function CreateCompatibleDC Lib "gdi32" (ByVal hdc As Long) As Long

Declare Function SelectObject Lib "gdi32" (ByVal hdc As Long, ByVal hObject As Long) As Long

Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long

Declare Function OpenClipboard Lib "user32" (ByVal hwnd As Long) As Long

Declare Function EmptyClipboard Lib "user32" () As Long

Declare Function SetClipboardData Lib "user32" (ByVal wFormat As Long, ByVal hMem As Long) As Long

Declare Function CloseClipboard Lib "user32" () As Long

Declare Function DeleteDC Lib "gdi32" (ByVal hdc As Long) As Long

Declare Function ReleaseDC Lib "user32" (ByVal hwnd As Long, ByVal hdc as Long) As Long

Public Const SRCCOPY = &HCC0020 ' (DWORD) dest = source

Public Const CF_BITMAP = 2
 

Sub ScreenToClipBoard()

Dim bHwnd As Long

Dim hdc As Long

Dim hdcMem As Long

Dim hBitmap As Long
 

Dim RECT As RECT

Dim junk As Long

Dim fwidth As Long

Dim fheight As Long

Dim x As Long

Dim y As Long

Dim lColor As Long
 

    '---------------------------------------------------

    ' Your code to open a browser window and navigate to 

    ' the site goes here

    '---------------------------------------------------

 

    '---------------------------------------------------

    ' Get window handle to Window

    '---------------------------------------------------

    ' I have Mozilla on that particular machine, use whatever works for you

    bHwnd = FindWindow("MozillaWindowClass", "Welcome to the Cook County Assessor's Virtual Office - Mozilla")
 

    '---------------------------------------------------

    ' Get the device context of the window and allocate memory

    '---------------------------------------------------
 

    hdc = GetDC(bHwnd)

    hdcMem = CreateCompatibleDC(hdc)

    hBitmap = CreateCompatibleBitmap(hdc, 86, 21)

    If hBitmap <> 0 Then

       junk = SelectObject(hdcMem, hBitmap)

    Else

       MsgBox "Couldn't create mem bitmap"

       Exit Sub

    End If

    

    '---------------------------------------------------

    ' Get screen dimensions of the browser window

    '---------------------------------------------------

    Call GetWindowRect(bHwnd, RECT)

    fwidth = RECT.Right - RECT.Left

    fheight = RECT.Bottom - RECT.Top
 

    '---------------------------------------------------

    ' Look at each pixel for the color behind the sec code

    '---------------------------------------------------

 

    For y = 0 To fheight

        For x = 0 To fwidth

            lColor = GetPixel(hdc, x, y)

            If lColor = 14540253 Then

                BitBlt hdcMem, 0, 0, 86, 21, hdc, x, y, SRCCOPY

                '---------------------------------------------

                ' Set up the Clipboard and copy bitmap

                '---------------------------------------------

                junk = OpenClipboard(bHwnd)

                junk = EmptyClipboard()

                junk = SetClipboardData(CF_BITMAP, hBitmap)

                junk = CloseClipboard()

                GoTo CleanUp

            End If

        Next x

    Next y

 

CleanUp:

    '---------------------------------------------

    ' Clean up handles

    '---------------------------------------------

    junk = DeleteDC(hdcMem)

    junk = ReleaseDC(bHwnd, hdc)

    Beep

    

 End Sub

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20330919
Mike,
Thanks for the second update. I was able to only spend a couple of hours on the code in your first suggestion and I didn't have much positive luck. I won't go into the details now since I just got your second suggestion.

I really appreciate you going "above and beyond" by taking the time to go to the web site and use SPY++ on it. I was able to get a copy of 'Winspector' from the site suggested by Idle Mind. It shows lots of interesting stuff and hopefully in time, I'll be able to understand what I'm looking at a little better.

I'll probably continue to work on this over the long weekend, but I hope that you enjoy the long Thanksgiving weekend.

John
0
 
LVL 18

Expert Comment

by:mdougan
ID: 20335437
Thanks John,

Hope you have some success with it.  I'm not sure if my approach was clear in the above code, but basically what I was trying to do was capture the entire browser window as a bitmap, then, search each pixel looking for a color that matches the background color of the security code area.  That should give you the upper left corner of that area.  Then, we already know the width and height, so, we could copy just that section of the bitmap to a new bitmap.  The problem is that the GetPixel function works well if the bitmap is loaded into a PictureBox control, however, it doesn't seem to work with the kind of bitmap that our screenshot was loaded into.

There might be some other object we could copy the screenshot into, such as a Standard Picture object, and perhaps GetPixel would work with that.  Things to consider if you're searching through code examples this weekend.

Mike
0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20352793
Mike,

I understand your approach and it looks like a logical way to do what we want to do. However, when I test my code the only value returned from the GetPixel function is a -1. I think this means we're not looking at what we think we're looking at.

Unfortunately, I didn't get the time I had hoped for over the long weekend to work on this and it looks like I've got a few other fires that need my attention during the first day or so of this week.

I noticed a 'ScreentoClipBoard' function earlier and I'll try to do something with it when time is available. If you think of anything, please let me know.

A copy of my code at this point is included.

Hope you enjoyed your weekend.

John
    IEHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE

    If IEHwnd = 0 Then Stop  ' Did not find Window with the "WindowTitleText".
 

    ' Get the Device context of the window and allocate memory

    HDC = GetDC(IEHwnd)

    HDCmem = CreateCompatibleDC(HDC)

    hBitMap = CreateCompatibleBitmap(HDC, 86, 21)

    If hBitMap <> 0 Then

        junk = SelectObject(HDCmem, hBitMap)

        Else

        Stop ' Couldn't create mem bitmap

        Exit Sub

    End If

    

    ' Get screen dimensions of IE Window

    Call GetWindowRect(IEHwnd, Rect)          ' Get the borders of the IE Window.

    fwidth = Rect.Right - Rect.Left

    fheight = Rect.Bottom - Rect.Top

    

    success = False

    For y = 0 To fheight

        For x = 0 To fwidth

        lcolor = GetPixel(HDC, x, y)

        If lcolor <> -1 Then Stop

        If lcolor = 14540253 Then

            BitBlt HDCmem, 0, 0, 86, 21, HDC, x, y, SRCCOPY

            success = True

            Exit For

        End If

        Next x

    Next y

    

    If success = True Then Stop

Open in new window

0
 
LVL 18

Expert Comment

by:mdougan
ID: 20361022
By George, I think I've got it :-)

I found out that for GetPixel to work, you have to use the SelectObject on the hDC and a bitmap first.

So, I modified the code a little bit:

First, I get the browsers window handle using FindWindow

Next, I found it was necessary to bring the browser window to the foreground, otherwise overlapping window images will be included in the BitBlt.  Since it takes a second for a window to move to the foreground, I did a few DoEvents to give it enough time.

Get the dimensions of the source browser window into the Rect

Get the hDC for the browser window and create a compatible bitmap for it

Use SelectObject on the browser window/bitmap

Create a target hDC and bitmap

Search the source bitmap using GetPixel, looking for the backcolor of the security code

When found, select the target bitmap into the target hDC using SelectObject

BitBlt the source bitmap into the target

Copy the target bitmap into the Clipboard

In this example, I paste the clipboard into the first cell of the active spreadsheet

Important note, when cleaning up the handles, be sure to delete the bitmaps, otherwise you'll have a large memory leak!  You might want to search other examples and ensure that you are deleting/releasing all the necessary objects and handles, as a memory leak is bad news!

This code sample assumes that you've already created the browser window and navigated to the URL and expanded one of the rows to expose the security code.  Also make sure to substitute your FindWindow code for the code in the example that is only valid for my Mozilla browser.

Whew!
Public Type RECT

        Left As Long

        Top As Long

        Right As Long

        Bottom As Long

End Type
 

Declare Function GetPixel Lib "gdi32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long) As Long

Declare Function FindWindow Lib "user32" Alias "FindWindowA" (ByVal lpClassName As String, ByVal lpWindowName As String) As Long

Declare Function GetWindowRect Lib "user32" (ByVal hwnd As Long, lpRect As RECT) As Long

Declare Function GetDC Lib "user32" (ByVal hwnd As Long) As Long

Declare Function CreateCompatibleBitmap Lib "gdi32" (ByVal hDC As Long, ByVal nWidth As Long, ByVal nHeight As Long) As Long

Declare Function CreateCompatibleDC Lib "gdi32" (ByVal hDC As Long) As Long

Declare Function SelectObject Lib "gdi32" (ByVal hDC As Long, ByVal hObject As Long) As Long

Declare Function BitBlt Lib "gdi32" (ByVal hDestDC As Long, ByVal x As Long, ByVal y As Long, ByVal nWidth As Long, ByVal nHeight As Long, ByVal hSrcDC As Long, ByVal xSrc As Long, ByVal ySrc As Long, ByVal dwRop As Long) As Long

Declare Function OpenClipboard Lib "user32" (ByVal hwnd As Long) As Long

Declare Function EmptyClipboard Lib "user32" () As Long

Declare Function SetClipboardData Lib "user32" (ByVal wFormat As Long, ByVal hMem As Long) As Long

Declare Function CloseClipboard Lib "user32" () As Long

Declare Function DeleteDC Lib "gdi32" (ByVal hDC As Long) As Long

Declare Function ReleaseDC Lib "user32" (ByVal hwnd As Long, ByVal hDC As Long) As Long

Declare Function DeleteObject Lib "gdi32" (ByVal hObject As Long) As Long
 

Declare Function SetForegroundWindow Lib "user32" (ByVal hwnd As Long) As Long
 

Public Const SRCCOPY = &HCC0020 ' (DWORD) dest = source

Public Const CF_BITMAP = 2
 
 

Sub ScreenToClipBoard()

Dim bHwnd As Long

Dim hDC As Long

Dim hBitmap As Long

Dim hdcMem2 As Long

Dim hBitmap2 As Long
 

Dim RECT As RECT

Dim junk As Long

Dim fwidth As Long

Dim fheight As Long

Dim x As Long

Dim y As Long

Dim lColor As Long
 

    '---------------------------------------------------

    ' Insert your code here to create the browser window

    ' and navigate to your URL

    '---------------------------------------------------
 

    '---------------------------------------------------

    ' Get window handle to Browser

    ' change this as necessary to work with your browser

    '---------------------------------------------------

    bHwnd = FindWindow("MozillaWindowClass", "Welcome to the Cook County Assessor's Virtual Office - Mozilla")

    

    ' The browser window has to be on top of all other windows, otherwise their images will overlay the browser window's

    junk = SetForegroundWindow(bHwnd)

    

    ' There will be a lag before the window is actually in the foreground, so, give it a little CPU to move to the front

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    

    '---------------------------------------------------

    ' Get screen coordinates of the browser window

    '---------------------------------------------------

    Call GetWindowRect(bHwnd, RECT)

    fwidth = RECT.Right - RECT.Left

    fheight = RECT.Bottom - RECT.Top

        

    '---------------------------------------------------

    ' Get a handle to the browser window

    '---------------------------------------------------

    hDC = GetDC(bHwnd)

    

    '---------------------------------------------------

    ' GetPixel only works if the hDC and bitmap have been selected with SelectObject

    ' So, create a compatible bitmap and select it

    '---------------------------------------------------

    hBitmap = CreateCompatibleBitmap(hDC, fwidth, fheight)

    If hBitmap <> 0 Then

       junk = SelectObject(hDC, hBitmap)

    Else

       MsgBox "Couldn't create mem bitmap"

       Exit Sub

    End If
 

    '---------------------------------------------------

    ' Now, create an hDC and Bitmap for the final result bitmap

    '---------------------------------------------------

    hdcMem2 = CreateCompatibleDC(hDC)

    hBitmap2 = CreateCompatibleBitmap(hDC, 86, 21)

            

    '---------------------------------------------------

    ' Search the browser window looking for the backcolor

    ' of the security code image

    '---------------------------------------------------

    For x = 0 To fheight - 1

        For y = 0 To fwidth - 1

            lColor = GetPixel(hDC, x, y)

            If lColor = 14540253 Then

                '---------------------------------------------

                ' Once the security code is found, prepare to copy it

                ' to another memory resident bitmap

                '---------------------------------------------

                

                If hBitmap2 <> 0 Then

                  junk = SelectObject(hdcMem2, hBitmap2)

                Else

                  MsgBox "Couldn't create mem bitmap"

                  Exit Sub

                End If
 

                BitBlt hdcMem2, 0, 0, 86, 21, hDC, x, y, SRCCOPY

                

                '---------------------------------------------

                ' Set up the Clipboard and copy memory resident bitmap to it

                '---------------------------------------------

                junk = OpenClipboard(bHwnd)

                junk = EmptyClipboard()

                junk = SetClipboardData(CF_BITMAP, hBitmap2)

                junk = CloseClipboard()

                

                '---------------------------------------------

                ' Optionally, paste the clipboard contents into a cell on the spreadsheet

                '---------------------------------------------

                Range("A1").Select

                ActiveSheet.Paste

                

                GoTo CleanUp

            End If

        Next y

    Next x

 

CleanUp:

    '---------------------------------------------

    ' Clean up handles, important, don't forget to

    ' delete the bitmaps or you will have a memory leak!

    '---------------------------------------------

    junk = DeleteObject(hBitmap)

    junk = DeleteObject(hBitmap2)

    junk = DeleteDC(hdcMem2)

    junk = ReleaseDC(bHwnd, hDC)

    

 End Sub

 

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20365790
Mike,

Your last response made my day !! , Yes, it can be done !!!. I changed my code and unfortunately, it still doesn't work for me. I added a few lines of debugging code to make sure that when I try to use the 'SelectObject' that it returns a non zero value. It doesn't, I always get zero.

Do we possibly have a problem that I am using IE and you are using Mozilla or is it something else that I just can't find in my code? I would appreciate you looking at the snippet to see if I'm missing something somewhere. If it is an incompatibility between IE/Mozilla, I'm certainly not adverse to using Mozilla if that what it takes to make it work. However, since I never used it before I want to proceede with caution to make sure nothing else is adversely affected.

Thanks again for your support and patience. If you're ever in Chicago, the drinks are on me.

John


 
Private Sub ScreenToClipBoard()

Dim bHwnd As Long

Dim fwidth As Long

Dim fheight As Long

Dim hBitMap As Long

Dim hBitMap2 As Long

Dim hDC As Long

Dim hDCmem2 As Long

Dim lColor As Long

Dim junk As Long

Dim Rect As Rect

Dim x As Long

Dim y As Long
 
 

Stop

    bHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE

    If bHwnd = 0 Then

        MsgBox ("Did not find the Window with the title of:" & vbCrLf & vbCrLf & _

                """" & WindowTitleText & """" & vbCrLf & vbCrLf & _

                "Error number=" & Err.LastDllError)

        Exit Sub

    End If
 
 

    junk = SetForegroundWindow(bHwnd)   ' Set this handle/Window as the Foreground Window.

    If junk = 0 Then                    ' A zero return code indicates failure

        MsgBox ("Unable to Set IE as the foreground Window.")

        Exit Sub

    End If

    

    '---------------------------------------------------

    ' There will be some lag before this window is actually in the foreground, so, give it a

    ' little CPU to move it to the front

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    

    '---------------------------------------------------

    ' Get screen coordinates of the browser window

    '---------------------------------------------------

    Call GetWindowRect(bHwnd, Rect)

    fwidth = Rect.Right - Rect.Left         ' Calculate Width of Window

    fheight = Rect.Bottom - Rect.Top        ' Calculate Height of Window

    

    '---------------------------------------------------

    ' Get a handle to the browser window

    '---------------------------------------------------

    hDC = GetDC(bHwnd)                      ' Get 'Device Context' for IE window.

    

    '---------------------------------------------------

    ' GetPixel only works if the hDC and the bitmap have been selected with SelectObject

    ' So, create a compatible bitmap and select it.

    '---------------------------------------------------

    hBitMap = CreateCompatibleBitmap(hDC, fwidth, fheight)

    If hBitMap <> 0 Then

        junk = SelectObject(hDC, hBitMap)

        If junk = 0 Then                        ' In my testing I always get a zero

            MsgBox ("Unable to SelectObject")   ' See http://msdn2.microsoft.com/en-us/library/ms533272.aspx

            Exit Sub

        End If

    Else

        MsgBox "Could NOT create a memory bitmap"

        Exit Sub

    End If

    

    '----------------------------------------------------

    ' Now create a hDC and bitmap for the final result bitmap.

    '----------------------------------------------------

    hDCmem2 = CreateCompatibleDC(hDC)

    hBitMap2 = CreateCompatibleBitmap(hDC, 86, 21)

    

    '----------------------------------------------------

    ' Search the browser window looking for the backcolor

    ' of the security code image

    '----------------------------------------------------

    For x = 0 To fheight - 1

        For y = 0 To fwidth - 1

            lColor = GetPixel(hDC, x, y)

            If lColor <> -1 Then Stop       ' My testing never stop here

            If lColor = 14540253 Then

                '---------------------------------------------

                ' Once the security code is found, prepare to copy it

                ' to another memory resident bitmap

                '---------------------------------------------

                

                If hBitMap2 <> 0 Then

                    junk = SelectObject(hDCmem2, hBitMap2)

                Else

                    MsgBox ("Could NOT create memory bitmap")

                    Exit Sub

                End If

            

                BitBlt hDCmem2, 0, 0, 86, 21, hDC, x, y, SRCCOPY

                

                '---------------------------------------------

                ' Set up the Clipboard and copy memory resident bitmap to it

                '---------------------------------------------

                junk = OpenClipboard(bHwnd)

                junk = EmptyClipboard()

                junk = SetClipboardData(CF_BITMAP, hBitMap2)

                junk = CloseClipboard()

        

                '---------------------------------------------

                ' Optionally, paste the clipboard contents into a cell on the spreadsheet

                '---------------------------------------------

                Range("A1").Select

                ActiveSheet.Paste

                

                '---------------------------------------------

                ' Clean up handles

                '---------------------------------------------

                junk = DeleteObject(hBitMap)

                junk = DeleteObject(hBitMap2)

                junk = DeleteDC(hDCmem2)

                junk = ReleaseDC(bHwnd, hDC)

            End If

        Next y

    Next x
 

Stop

 

End Sub

Open in new window

0
 
LVL 18

Expert Comment

by:mdougan
ID: 20367278
Hi,

No, I don't think it would have anything to do with which browser we're using... the only thing that should be affected by that is the FindWindow, and as long as you are getting back a valid window handle from that call, you should be OK.  

More likely is that there may be a difference in our operating systems.  The computer that I have Excel on is pretty old, and I'm running Windows NT on that machine.  I know that some of these low level graphics routines changed under XP, so, it is possible that things which work under NT will not work under XP.

If GetPixel is always returning a -1, then that means that the hDC is not properly selected.

I went to the link you provided in the code and noticed a statement that said that bitmaps can only be selected for memory DCs.  I'd tried to be efficient and just use the hDC of the physical browser window, since we have it.  However, that may not be considered a memory DC.  So, I've taken your code and I've created another compatible hDCmem, selected the hBitMap into the hDCmem and then did a BitBlt of the browser window into that hDCmem.  Then, everywhere below that line of code, where hDC was referenced, I changed the code to reference the hDCmem.  I wasn't sure if the BitBlt should come before or after the SelectObject, I guessed after, but if it doesn't seem to work, then you might try it before.

I'm not on the computer that has Excel, so, I can't test it, but give it a try and let me know how it goes!

I'll be happy to take you up on your offer if I get out to Chicago sometime.  I lived in Chicago for a while, a few years back.  It's a great place.  But I'm in NY now.
Private Sub ScreenToClipBoard()

Dim bHwnd As Long

Dim fwidth As Long

Dim fheight As Long

Dim hBitMap As Long

Dim hBitMap2 As Long

Dim hDC As Long

Dim hDCmem As Long

Dim hDCmem2 As Long

Dim lColor As Long

Dim junk As Long

Dim Rect As Rect

Dim x As Long

Dim y As Long

 

 

    bHwnd = FindWindow(vbNullString, WindowTitleText)  ' Find Handle for IE

    If bHwnd = 0 Then

        MsgBox ("Did not find the Window with the title of:" & vbCrLf & vbCrLf & _

                """" & WindowTitleText & """" & vbCrLf & vbCrLf & _

                "Error number=" & Err.LastDllError)

        Exit Sub

    End If

 

 

    junk = SetForegroundWindow(bHwnd)   ' Set this handle/Window as the Foreground Window.

    If junk = 0 Then                    ' A zero return code indicates failure

        MsgBox ("Unable to Set IE as the foreground Window.")

        Exit Sub

    End If

    

    '---------------------------------------------------

    ' There will be some lag before this window is actually in the foreground, so, give it a

    ' little CPU to move it to the front

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    DoEvents

    

    '---------------------------------------------------

    ' Get screen coordinates of the browser window

    '---------------------------------------------------

    Call GetWindowRect(bHwnd, Rect)

    fwidth = Rect.Right - Rect.Left         ' Calculate Width of Window

    fheight = Rect.Bottom - Rect.Top        ' Calculate Height of Window

    

    '---------------------------------------------------

    ' Get a handle to the browser window

    '---------------------------------------------------

    hDC = GetDC(bHwnd)                      ' Get 'Device Context' for IE window.

    

    '---------------------------------------------------

    ' GetPixel only works if the hDC and the bitmap have been selected with SelectObject

    ' So, create a compatible bitmap and select it.

    '---------------------------------------------------

    hDCmem = CreateCompatibleDC(hDC)

    hBitMap = CreateCompatibleBitmap(hDC, fwidth, fheight)
 

    ' Not sure if this should be before or after the SelectObject

    'BitBlt hDCmem, 0, 0, fwidth, fheight, hDC, 0, 0, SRCCOPY
 
 

    If hBitMap <> 0 Then

        junk = SelectObject(hDCmem, hBitMap)

        If junk = 0 Then                        ' In my testing I always get a zero

            MsgBox ("Unable to SelectObject")   ' See http://msdn2.microsoft.com/en-us/library/ms533272.aspx

            Exit Sub

        End If

    Else

        MsgBox "Could NOT create a memory bitmap"

        Exit Sub

    End If

   

    ' Not sure if this should be before or after the SelectObject

    BitBlt hDCmem, 0, 0, fwidth, fheight, hDC, 0, 0, SRCCOPY
 

 

    '----------------------------------------------------

    ' Now create a hDC and bitmap for the final result bitmap.

    '----------------------------------------------------

    hDCmem2 = CreateCompatibleDC(hDC)

    hBitMap2 = CreateCompatibleBitmap(hDC, 86, 21)

    

    '----------------------------------------------------

    ' Search the browser window looking for the backcolor

    ' of the security code image

    '----------------------------------------------------

    For x = 0 To fheight - 1

        For y = 0 To fwidth - 1

            lColor = GetPixel(hDCmem, x, y)

            If lColor = 14540253 Then

                '---------------------------------------------

                ' Once the security code is found, prepare to copy it

                ' to another memory resident bitmap

                '---------------------------------------------

                

                If hBitMap2 <> 0 Then

                    junk = SelectObject(hDCmem2, hBitMap2)

                Else

                    MsgBox ("Could NOT create memory bitmap")

                    Exit Sub

                End If

            

                BitBlt hDCmem2, 0, 0, 86, 21, hDCmem, x, y, SRCCOPY

                

                '---------------------------------------------

                ' Set up the Clipboard and copy memory resident bitmap to it

                '---------------------------------------------

                junk = OpenClipboard(bHwnd)

                junk = EmptyClipboard()

                junk = SetClipboardData(CF_BITMAP, hBitMap2)

                junk = CloseClipboard()

        

                '---------------------------------------------

                ' Optionally, paste the clipboard contents into a cell on the spreadsheet

                '---------------------------------------------

                Range("A1").Select

                ActiveSheet.Paste

                

                '---------------------------------------------

                ' Clean up handles

                '---------------------------------------------

                junk = DeleteObject(hBitMap)

                junk = DeleteObject(hBitMap2)

                junk = DeleteDC(hDCmem)

                junk = DeleteDC(hDCmem2)

                junk = ReleaseDC(bHwnd, hDC)

            End If

        Next y

    Next x

 

Stop

 

End Sub

Open in new window

0
 
LVL 1

Author Comment

by:PropertyTaxAnalysts
ID: 20368751
Mike,
Wow, it works great!!! I still have to do some more code to make sure I handle the bitmaps correctly but ALL the points will go to you. I am extremely grateful for your help, support and encouragement on this. I thought it could be done this way, and with your expertise you made it happen.

I'll post here again if I run into other problems related to this and also to notify you when I sure everything works the way it should.

Thanks again,

John
0
 
LVL 18

Accepted Solution

by:
mdougan earned 500 total points
ID: 20373516
My pleasure. I enjoy a challenge and this was a good one!

Some people just want you to hand them a completed solution, but I prefer working "with" the person asking the question.  You are great that way.

Just comment here if you open up another item you want me to look at and I'll be notified.  And close this item whenever you are satisfied with it.

Cheers!
0
 
LVL 1

Author Closing Comment

by:PropertyTaxAnalysts
ID: 31409174
This was a difficult problem and mdougan did an excellent job in helping me develop a solution and more importantly, he took the time and had enough patience to make sure I understood the solution also.

He's the kind of guy you want to be sure you keep as a resource!
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
MS Access - Capture pressed key onclick 4 21
Program tries to re-install when opening 6 53
Copy a row 12 53
active directory 5 48
Windows Script Host (WSH) has been part of Windows since Windows NT4. Windows Script Host provides architecture for building dynamic scripts that consist of a core object model, scripting hosts, and scripting engines. The key components of Window…
I was working on a PowerPoint add-in the other day and a client asked me "can you implement a feature which processes a chart when it's pasted into a slide from another deck?". It got me wondering how to hook into built-in ribbon events in Office.
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now