Solved

reading screen caracters

Posted on 1999-01-22
35
349 Views
Last Modified: 2012-06-21
I would like to have the code to search the sreen for a certain words for example "name:" then read the next 10 characters and put this in a variable ex:"name:paul" i want paul in a variable.  I would have this program running in background and when i hit a certain key i want it to search any windows program that is up on screen for that word. I asume that low level api would have to be used for this.  I had done this long time ago in asembler. Lots a point on this one!!!
0
Comment
Question by:jerbell
  • 11
  • 8
  • 7
  • +4
35 Comments
 
LVL 12

Expert Comment

by:mark2150
Comment Utility
Problem is that unlike the old DOS days, the screen doesn't exist as characters. You can read in display memory alright, and what you'll get is lots of *pixels* - NO ASCII!

Problem becomes *HUGE* in a hurry as you you effectively have to OCR the screen using every possible system font. *I* wouldn't want to code it.

I'd like the points. My guess is that it's a NCD (No Can Do)

You'ld have to intercept the messages as they were *sent* to the windows, not scan the screen memory once they've been rendered.

M

0
 
LVL 4

Expert Comment

by:chabaud
Comment Utility
I am not sure mark understand the question.

I have vb code (with API) to scan all the windows and retreive text on them. So it is easy to search a given string within this text.


0
 

Author Comment

by:jerbell
Comment Utility
NCD is not good enuf *l*
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
I understand perfectly what you want to do. Here is the best I can think that remotely aproximates to your need. As mark2150 said, once they are rendered, there is no way to get them from the screen, unless you have OCR (and a very good one). So, my first answer is: I'm sorry, you cannot do that.

But since I'd like to get those points, I'll give you a few tips I can think of.

You could try to walk all the window hierachy, starting with the Desktop, and calling the GetWindow Win32 API o it to get siblings, children and so on. For each window you could do a GetWindowText(), and like this you can get the words that are in captions of every window.

Within this loop, find using GetWindowClass if the window is either a ListBox or a ComboBox, and you can retrieve the text of all items using the corresponding control messages. Although you could do that for listview and treeview controls (and the other common controls), Windows will not provide default marshalling for memeory from the window's process to your process, and that is a lot of work (see Win32 Q&A, MSJ, Sep 97 for info on that).

That will do it for text that is contained by a window (captions, listbox items, etc.) But some other text is drawn directly to the screen, with calls to Win32 APIs like TextOut, DrawText, and so on. You could hook those APIs globally, and see what strings are to be drawn, but I know of no method to hook APIs globally in every process on any DLL that work in a general way. Maybe you could hack something that is ok for your needs. But in case you managed to hook those APIs, you would also need to determine when something else was drawn over that text, to remove it from your list of words (maybe by hooking WM_PAINT messages with a WH_GETMESSAGE hook and looking at the coordinates of the invalid region in the passed PAINTSTRUCT).

That is all I can think of. The solution is far from perfect, but if your needs are not very big, it might work.

I'd recommend you do it (in case you decide to do what I said) in C/C++, since they allow easier calling of the Win32 API.

In conclusion, there is no fool-proof way to do what you want. Sorry!

  -- LuisR

P.S. If you need any comments, help, whatever, you can write me to luisr@iname.com.

0
 

Author Comment

by:jerbell
Comment Utility
It's not exactly what i was looking for *sorry*
I would love to give you a few points for your answer though cause it was a bit meaningful
but i doin't know how to do that.
I guess my question might not have a solution.
*sigh*

0
 

Author Comment

by:jerbell
Comment Utility
I kinda was searching for chabaud solution!!!
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
I think chabaud's solution is what I described about enumeratiang all windows and calling GetWindowText on them.

Sorry, but what you want to do simply cannot be done. But, what are you trying to accomplish? Maybe there are better and easier ways to do it.

0
 
LVL 4

Expert Comment

by:chabaud
Comment Utility
Here is the code with using GetWindow and GetWindowText:

Option Explicit

Private sKey As String
Private sVal As String

Private Declare Function EnumWindows Lib "user32" (ByVal lpEnumFunc As Long, ByVal lParam As Long) As Long
Private Declare Function GetWindowTextLength Lib "user32" Alias "GetWindowTextLengthA" (ByVal hwnd As Long) As Long
Private Declare Function GetWindowText Lib "user32" Alias "GetWindowTextA" (ByVal hwnd As Long, ByVal lpString As String, ByVal cch As Long) As Long
Private Declare Function GetWindow Lib "user32" (ByVal hwnd As Long, ByVal wCmd As Long) As Long
Private Const GW_CHILD = 5
Private Const GW_HWNDNEXT = 2

Public Function SearchIt(ByRef s As String) As String

    sKey = s
    sVal = ""
   
    EnumWindows AddressOf WndEnumProc, 0
   
    SearchIt = sVal

End Function

Private Function WndEnumProc(ByVal hwnd As Long, ByVal lParam As Long) As Long

    Dim szText As String
    Dim hWndChild As Long
    Dim c As Integer
   
    szText = String(512, 0)
   
    If GetWindowText(hwnd, szText, 512) Then
   
        c = InStr(szText, sKey)
        If c Then
            sVal = Mid(szText, c + Len(sKey), 40)
            'lParam.AddItem Mid(szText, c, 20)
        End If
    End If
   
    'Recursively find child windows
    hWndChild = GetWindow(hwnd, GW_CHILD)
    Do While hWndChild
        WndEnumProc hWndChild, 0
       
        hWndChild = GetWindow(hWndChild, GW_HWNDNEXT)
    Loop
   
    WndEnumProc = 1

End Function


0
 
LVL 3

Expert Comment

by:covington
Comment Utility
If you know in advance what type of applications you may be searching, it would make it easier to write the code.
0
 

Author Comment

by:jerbell
Comment Utility
This is the deal!! I have a user that runs a communication package called keaterm
They used this package to log on to server via network and run the server application.
This application has employee#:12345
What i want to do is when user hit hotkey, i want to search the keaterm session for employee#: then put 12345 in variable then go to a picture server and open 12345.jpg
Now the application that i run in keaterm is caracter based so theres no list box combo and all that fancy stuff.
0
 
LVL 15

Expert Comment

by:ameba
Comment Utility
Here is the code. I tested it with NC (don't have NCD)
Start new project, and paste code to your Form1.

' Form1
' add 2 command buttons and 1 textbox (multiline=true)
Option Explicit

Private Sub Form_Load()
    Command1.Caption = "Get Text"
    Command2.Caption = "Find 'Name:'"
    Text1.Font = "Fixedsys"
End Sub

Private Sub Command1_Click()
    On Error GoTo gettext_Err
    Dim tosend As String
    Dim WinCaption As String ' caption of your DOS window
    ' make sure it is not running Full Screen, and it is not minimized
    WinCaption = "Norton Commander - NCMAIN" ' use your own
   
    AppActivate WinCaption ' activate DOS window
   
    tosend = "% ek{Right 0}{Down 0}+({Right 79}{Down 24})"
    ' or this can grab smaller portion of screen
    'tosend = "% ek{Right 2}{Down 2}+({Right 59}{Down 18})"
    SendKeys tosend, 1 ' mark
    SendKeys "{Enter}", 1 ' copy
   
    If Clipboard.GetFormat(vbCFBitmap) Or Clipboard.GetFormat(vbCFDIB) Then
        ' show grabbed image (if DOS works in graphical mode)
        Me.Picture = Clipboard.GetData()
    ElseIf Clipboard.GetFormat(vbCFText) Then
        ' show grabbed text in textbox
        Text1.Text = Clipboard.GetText()
    End If
    Exit Sub
gettext_Err:
    MsgBox Error
    Exit Sub
End Sub

Private Sub Command2_Click()
    Dim sToFind As String, sGrabbed As String, pos As Integer
    sGrabbed = Text1.Text
    sToFind = "autoexec bat" ' mine
    'sToFind = "name:" ' yours
    pos = InStr(1, sGrabbed, sToFind, vbTextCompare) ' case insensitive
    If pos = 0 Then
        MsgBox sToFind & " not found!"
        Exit Sub
    End If
    Dim sNext10 As String
    sNext10 = Mid$(sGrabbed, pos + Len(sToFind), 10)
    Caption = "next 10 characters: " & sNext10
End Sub

0
 
LVL 4

Expert Comment

by:chabaud
Comment Utility
More explicit your are, more precise is the answer...
0
 

Author Comment

by:jerbell
Comment Utility
Well i increased the points to 700. Cause i would really like the code for this!!
Ameba i almost gave 500 to you cause i did try your answer in a dos based program and It seem to work relatively good but when i tried it on window based program it didn't work!!
Maybe I didn't explain myself good enuff!  My application is windowed base.  It's called keaterm 420 for win95.  But when i log on to server with this package i execute an informix program on Unix box.  So basicaly what i see in my windows keaterm session is caracter based and that is the text i want to capture!! Ameba if you can modify the code to work for windows program i will give you points or anybody else that can do this!!
Thanks!!!
0
 
LVL 15

Expert Comment

by:ameba
Comment Utility
Wow, 700 pts.
It looks like console application. Any more info (or surprise :-) )?
You have a prompt> ?
Caption?
0
 
LVL 15

Expert Comment

by:ameba
Comment Utility
Wow, 700 pts.
It looks like console application. Any more info (or surprise :-) )?
You have a prompt> ?
Caption?
0
 
LVL 3

Expert Comment

by:covington
Comment Utility
Jerbell,

Is this app (keaterm 420 for win95) public domain so I can test my solution on it?


0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
Try copying the whole terminal to the clipboard (bye sending a Select All, Copy commands) and then search on this text on the clipboard. That should be enough for your needs.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 4

Expert Comment

by:chabaud
Comment Utility
Does "the text window" have an handle, is it a pure graphic window or a textbox style one ?

Try to use SPY.EXE or SPYXX.EXE to get information about the window (its handle, class name, ...)
0
 

Author Comment

by:jerbell
Comment Utility
ameba Yes this would be a consol application. I do get a unix loging prompt. When i log in to the unix box through keaterm i execute an informix application wich doesn't have any graphic.  Keaterm is used basicaly as terminal emulation!  Unfortunatly it's not public domain there is similar public domain windows terminal emulating program that would act similar to keaterm.  I use this software for the convinient way of mapping keyboard and so on.  Luir what you said about copying to clipboard might work but i have to tell you all am new to visual basic and if i can get code that works i will give happily away my points.  
:o)
0
 
LVL 3

Expert Comment

by:covington
Comment Utility
Everybody wants to get the 700 points, and lots of us know how to do it, but we need the window handle and/or class of your terminal application. If you can find that, several of us could give you working code.

0
 

Author Comment

by:jerbell
Comment Utility
Here is what i got on my keaterm window using spy:

handle: 00000560
caption "vdxiii - kea! 420"
class: KEA!
class styles= 00000008 CS_DBCLKS
Style 1DCF0000
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
Ok, here is another try.

Does the terminal have edit/select all or something like that? does it have edit/copy? If it does you can use something like:

SetForegroundWindow FindWindow("KEA!", NULL)
SendKeys "Ctrl+A"  ' I don't remember the exact syntax,
SendKeys "Ctrl+C"  ' look in help. Replace with Term's keys

The call OpenClipboard and GetClipboadData to retrieve the information and search in that text for the data you want.

Sorry about not giving SendKeys correct syntax or OpenClipboad and GetClipboadData parameters, but I'm away from my work computer and I don't have any help in here.

If you are interested, I can write a DLL in C++ that copies the data from that terminal to a buffer (or a VB string) and gives it back to you for use in Visual Basic. Send me an e-mail.

0
 

Author Comment

by:jerbell
Comment Utility
I was kinda looking for the code since am very new to vbasic!!
Now am not sure on how to use dll and stuff i got a lot of reading to do.  I look in my kea program:  I do have a in my menu i got to chose kea edit copy and kea edit past
but i don't got a copy all and in order to do a copy i got to drag my mouse and select all my text.  It does send the selected text in the clipboard when i do past.
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
Ok, If it does nor have a Select All command, is there a way to select text with the keyboard? something like shift+something or anything like that? If it does, I can send you some code.
0
 

Author Comment

by:jerbell
Comment Utility
Ok am changing my Question since i found a way with keaterm to copy exactly what i want into windows clipboard.  I will still keep 700 points.  IF you want the POINTS i want the CODE.  I want code that will work in vb4 (32) to go retrieve in the clipboard the 7 caracters pasted there by keaterm. It would be in format 1234567.  I want to add extension to it .jpg and open that file(wich would be on network drive).
So as soon as i execute the program it should automaticaly go in clipbard retrieve 1234567 and open file 1234567.jpg

0
 
LVL 3

Expert Comment

by:covington
Comment Utility
Surely you don't want something as simple as:

Open Clipboard.GetText & ".jpg" For Input As #1

(I can't remember if the clipboard object is available in VB4...)
0
 

Author Comment

by:jerbell
Comment Utility
Did I forget to mention am new to vbasic?
Well it could be that simple.
I want vbasic to show the photo either by executing 1234567.jpg (and it would spawn the windows default picture viewer associated with .jpg extension) or do it with picture box on a form (either way i would be happy)! But as soon as i execute the vbasic program i want to see the photo of the file in the clipboard.
This could be easy question am not sure but will happily give 700 points for answer
0
 
LVL 3

Accepted Solution

by:
covington earned 700 total points
Comment Utility
Put a picturebox control on a Form.


Private Sub Form_Load()

    Picture1.Picture = LoadPicture("<path to filename>" & Clipboard.GetText & ".jpg")

End Sub


0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
As you have changed the question, it is sooo easy, almost 700 points for free.
0
 
LVL 3

Expert Comment

by:covington
Comment Utility
If I receive points, I am willing to share them with both luisr and ameba, as they did so much earlier work on this problem.
0
 

Author Comment

by:jerbell
Comment Utility
I hope you will share points!!!
As the answer got easier at the end thanks to keaterm and it's extended macro capabilities. No wonder i payed like over 200 for it.
Anyway i would like to thank everybody for the effort on this question!
It's people like you that makes expert exchange worth while.

jerbell
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
Wow, this question is an excellent example of how trying to do something generically is very hard, but trying to explain more in detail the purpose can make it much easier because it is a more specific answer...

Congratulations on finally receiving an answer, jerbell! :-) I hope your app works fine now.

Covington: I don't know if points can be shared, but it's ok if they can't.
0
 
LVL 3

Expert Comment

by:luisr
Comment Utility
Wow, this question is an excellent example of how trying to do something generically is very hard, but trying to explain more in detail the purpose can make it much easier because it is a more specific answer...

Congratulations on finally receiving an answer, jerbell! :-) I hope your app works fine now.

Covington: I don't know if points can be shared, but it's ok if they can't.
0
 
LVL 3

Expert Comment

by:covington
Comment Utility
I've asked about sharing points in http://www.experts-exchange.com/Q.10122943
0
 
LVL 7

Expert Comment

by:linda101698
Comment Utility
covington,
I'll add some points to your account which you can use to post questions for the other experts.  Since the question was worth 700 points, I'll add 466 which is roughly 2/3 of the total points.  You can then post questions directed to the other experts.

Linda Gardner
Customer Service @ Experts Exchange  
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Introduction While answering a recent question about filtering a custom class collection, I realized that this could be accomplished with very little code by using the ScriptControl (SC) library.  This article will introduce you to the SC library a…
Enums (shorthand for ‘enumerations’) are not often used by programmers but they can be quite valuable when they are.  What are they? An Enum is just a type of variable like a string or an Integer, but in this case one that you create that contains…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now