Detecting Screen changes

Just starting VB programming and what I am trying to do is to have my VB program detect a change in a different window, do some processing, and maybe process some mouse clicks in the window.

Things I want to look for in the window can either be text or background changes.

The window I want to detect changes is a different process, unrelated to the VB program. It might be an .EXE or maybe even a JAVA app.

Can I do it?
TomSlickEnterprise EngineerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

TomSlickEnterprise EngineerAuthor Commented:
Edited text of question.
In order for your application to know anything about another process, your program would have to install a message hook into the Windows message system.  A hook would allow you to preview all messages sent to and from the other process.

In ordwr to do this, the hook procedure must reside in a standard DLL library.

(Un)Fortunately, Visual Basic cannot create standard DLL files, only OLE DLL files, whis will not work for this purpose.

The DLL would have to be created in C, C++ and called from VB.
TomSlickEnterprise EngineerAuthor Commented:
OK, I cannot in tap into a process' message system....

How about interperting the screen display? Like a 'screen rake' type function.
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

TomSlickEnterprise EngineerAuthor Commented:
An example of what I want to do would be to write a program that would play FreeCell.

It would analyze the board, place flags on the mines (right click) or claim the square is clear (left click). it would then need to 'read' the results given on the Freecell screen to determine how it will play next.
TomSlickEnterprise EngineerAuthor Commented:
oopps... not FreeCell... but WinMine

have the program read the winmine screen and mouse click appropriately.

you can do this but why - there are already several programs that will solve mine fields for you.

To do this you would have to capture th screen and analyse this by comparing what you see with know images (eg. you'd need an imagelist with a copy of every possible square - a 1,2,3,4,5,6,7,8,blank,flag,mine,clear,question mark) and you'd have to check the image you capture, and then you'd have to send mouse clicks tot he minewseeper window.

TomSlickEnterprise EngineerAuthor Commented:

Your comment is along the lines of what I need to learn to do.

Now the why...
First I did mention that playing winmine is just an example of what i want to do. It is not going to be the finished product.

I have a lot of programming experience but it is all in the likes of BASIC, RPG, PL/1, COBOL, BAL..etc.  I am new to programming in the windows GUI environment and VERY new to VB so anything i do will be a learning experience for me.

Could you tell me how to capture a square from the screen? and i will bump up the points and award it to you.

Here is a MS article which shows how to capture an image of the form.  Look at the routine called "CaptureActiveWindow."

"HOWTO: Capture and Print the Screen, a Form, or any Window"

If you prefer to use the mouse to select what window to capture, here is another article.

"HOWTO: Use SetCapture and WindowFromPoint to Select a Window"

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
TomSlickEnterprise EngineerAuthor Commented:
Thanks for your post.....
The first link is exactly what I wanted. I have made some minor modification to the sample code provide  to capture portions of the screen verses the entire windows.

One minor problem, I have the following code....

   dim pic as picture
   dim pic1 as picture

   set pic = CaptureActiveWindow(..)
   set pic1 = CaptureActiveWindow(..)

   if pic = pic1 then
   end if

In this code, the conditional tests as false. Any thoughts on this? and yes, for my testing purposes, the 2 captureactivewindows ARE actually immediately executed one after the other.  

( erick,  you will be getting points for your previous comment whether or not you have any thoughts here )

Any new contributors will also get points appropriately.

Thank you.
Picture variables are like handles, they do not represent anything about the actual appearance of the image.

The code:

Dim pic1 As Picture
Dim pic2 As Picture
Set pic1 = Picture1.Picture
Set pic2 = Form1.Picture

evaluates false because, even though the images are identical, they are refering to different objects.

The code:

Dim pic1 As Picture
Dim pic2 As Picture
Set pic1 = Picture1.Picture
Set pic2 = Picture1.Picture

evaluates true because pic1 and pic2 are refering to the same object.

I think you have to examine the pixels of the images to determine if they are different.  For example, performing a BitBlt of one image over the other with the correct raster operation can result in a difference image.
TomSlickEnterprise EngineerAuthor Commented:
Maybe you have a better way to do this....

I am trying to monitor a screen and have my program perform various clicks depending on the state of the screen.

I have one area on the screen that will contain 1 of 3 possible pictures. I want my program to key off of this area, perform some calcs, and then perform some mouse clicks.

I had hoped that I could babysit the program through the first iterations of changes to the 'key' area. capturing the different data to 3 different pictures in my form. Then set up a timed loop to periodically capture the 'key' area and compare it to the 3 stored images to determine what state the screen is in. I could then leave this program running unattended.

any thoughts?

I want my program to key off of tha
TomSlickEnterprise EngineerAuthor Commented:
Thanks for your help Erick
You're welcome.
Here is a real simple picture comparison routine.  It draws both pictures to a picturebox (one normal, one inverse).  If the pictures are identical, the resultant picture is black.  Looping through each pixel, we can test for a non-black pixel.

Obviously, this is only practical for small bitmaps or very fast computers.

Option Explicit

Private Function PixelCompare(pic1 As IPicture, pic2 As IPicture, _
                pb As PictureBox) As Boolean
    If (pic1.Handle = 0) Or (pic2.Handle = 0) Then
        PixelCompare = False
        Exit Function
    End If
    Dim dx As Long, dy As Long
    Dim i As Long, j As Long
    'Get the pixel size of one of the pictures
    dx = ScaleX(pic1.Width, vbHimetric, vbPixels)
    dy = ScaleY(pic1.Height, vbHimetric, vbPixels)
    With pb 'PictureBox
    .BorderStyle = 0 'None
    .ScaleMode = 3 'Pixel
    'Assuming the form is in twips mode:
    .Height = dy * Screen.TwipsPerPixelY
    .Width = dx * Screen.TwipsPerPixelX
    '.Visible = False
    .AutoRedraw = True
    'Draw the 2 pics, one normal, one inverse.
    'If images are same, result is black
    .PaintPicture pic1, 0, 0, , , , , , , vbSrcCopy
    .PaintPicture pic2, 0, 0, , , , , , , vbSrcInvert
    'Loop through each pixel looking
    ' for a non-black pixel
    For i = 0 To dx - 1
        For j = 0 To dy - 1
            If .Point(i, j) <> vbBlack Then
                PixelCompare = False
                .AutoRedraw = False
                Debug.Print "Pixel "; i; j; Hex(.Point(i, j))
                End With
                Exit Function
            End If
    .AutoRedraw = False
    End With
    'Pictures are same
    PixelCompare = True
End Function

Private Sub Command1_Click()
    Dim p1 As Picture, p2 As Picture
    'Load some pictures
    Set p1 = LoadPicture("c:\windows\circles.bmp")
    Set p2 = LoadPicture("c:\windows\houndstooth.bmp")
    'Test the function
    Debug.Print PixelCompare(p1, p2, Picture1)
End Sub
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic Classic

From novice to tech pro — start learning today.