Solved

Urgent: Screnn Text From Under Mouse Pointer

Posted on 2000-05-01
31
384 Views
Last Modified: 2013-11-20
Hi, It's any one can help me to get screen text from under mouse pointer? For example if you do a right click on this question, first word (Hi), there are some popup will show the word "Hi" as in the screen.

I need the answer urgently. Many thanks for any help.

VHOA
0
Comment
Question by:pvhoa
  • 11
  • 10
  • 3
  • +3
31 Comments
 
LVL 3

Expert Comment

by:luisr
ID: 2767017
There is no way that will work for every single word on your screen, because once they are displayed, they become a simple array of dots. Just to get an idea on what it would take, see my coment on the question below:

http://www1.experts-exchange.com/bin/ShowQ?qid=10120330
0
 

Author Comment

by:pvhoa
ID: 2767082
Thank you, luisr. The problem is that I saw a program that can do this work. This is an e-dictionary from China, if I remembered. Just click on every word directly on the screen, you will have the translation right away, no matter where is this word, IE5, or MS Word. And I tried a lot to find out how this works, but no results yet.

Looking forward for your help.

Thanks again.
0
 
LVL 3

Expert Comment

by:V_Bapat
ID: 2768514
Since you want to get the screen text under the mouse, you need to either already know or determine at runtime, exactly where(position) you are displaying the text. Then with the help of the point the mouse button is clicked, you can find out the text.

If you can give more details about what you have done and what you are doing, it will be good.

Vicky
0
 
LVL 1

Expert Comment

by:yarond
ID: 2768708
pvhoa, can it be that the e-dictionary program you mention actually had OCR capabilities? If it can for example recognize a word that is contained in a displayed bitmap, then that is the only option, since you don't have anything that holds the actual text as text...
0
 
LVL 3

Expert Comment

by:luisr
ID: 2769761
yarond is right. There is no way to get the word under the mouse that works on very single case (IE, Word, dialog controls, menus, and so on) without somehow determining the coordinates of the word (that's actually not so hard) and then using some kind of OCR on it.
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2770826
If the accessibility options are installed on the machine where you want to run this, you'll find that you can use the ACtive Accessibility interfaces to read text and have descriptive information about windows controls returned to you.   That won't work in all cases, but in many cases (eg, all the regular control cases, and properly-enabled apps, like Office) it'll be fine.

Look up IAccessible in MSDN. That'll get you into the Active Accessibility documentation, and you can roam around from there.

..B ekiM
0
 

Author Comment

by:pvhoa
ID: 2783306
Thank all for comments. But anyway, I would like to have a general solution for all cases (Or for almost all).

And, because that I have not many points, but I have a lot of questions, so I should close this question. I will transfer the points of this questions to another one.

Thanks for help.
VHOA
0
 

Author Comment

by:pvhoa
ID: 2783353
This question has a deletion request Pending
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2863375
This question no longer is pending deletion
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2863376
I object. You've gotten a bunch of different suggestions for solutions, and that was because you posted a lucrative amount of points. You said your question was "very urgent".

You should use the IAccessibility interfaces. That works in every responsibly written program--if a program doesn't support IAccessibility, disabled people can't use it even with the disability features installed in Windows.

Unfortunately, just the same, most programs don't support IAccessibility. So, for those apps, you'll need to use the OCR approach that's been offered to you here.

Now, you've benefited from the research and suggestions others have made, but are withdrawing the points without evaluating the work we've done for you. You said you'd post other questions with points, but you haven't. Your behaviour is wrong and dishonest, and exemplifies a lot of what's critically wrong with ExpertsExchange.

..B ekiM
0
 

Author Comment

by:pvhoa
ID: 2869911
Hi Mike,

I'm very sorry that I didn't express my mind clearly to you and the others what caused a lot of misunderstanding. I would like to comfirm you that I'm always honest and fair, and I trully feel sorry about your misunderstanding of my behaviors. I'll try to explain you the reasons of deleting this question:

1. After posting a question, I did some searches on Experts-Exchange trying to find the similar questions and answers. I found a dozen of them. And I spent some hundreds points to get these old answers. These answers covered all your comment here. But no one of them was fully satisfied me.

2. Since this question is critical for me to decide go/no-go, and I have no answer after timeout event, I put away this project, and forever. In the other side, I have not many points but I have a lot of questions I want to ask you for help, so delete this question is a natural mind, do you think so? If you don't let me delete the question, how can I get the points for new one?

3. Your comments and comments of the others is highly appreciated, but they didn't satisfy me at the end. You mentioned about the evaluation, but I don't know how can I evaluated posted comments. I think the only one I can do here is sharing with you what I got after researching this problem, do you think it's fair enough? OK, here is the comments and answer from Anderson Adrian, what I found a very smart solution (But anyway, did not satisfy me). I thought I have right to re-publish his answer, because I paid for this. Have a look:

"The difficulty you've run into exists because most programs' document areas aren't controls, so GetWindowText() doesn't work (you've already discovered this).  The document area may just as well have a picture as it does text.  Given that, I'd suggest looking into writing a hook that attaches to the process associated with the window, once you're in the same process space you can ask the process to paint to a CMetaFileDC rather than a standard DC.  Once you have the HENHMETAFILE you can iterate through the draw operations that happened by calling EnumEnhMetaFile().  Iterate for every EMREXTTEXTOUT record.  Fortunately part of the EMREXTTEXTOUT record is a bounding rectangle.  If the point clicked by the mouse is within this rect, then this is the record so extract the EMRTEXT and be on your merry way.

The devil then is in the details (which would take more than 100 points).
"

If you would like to get details, please see the question in MFC box, ID: 10308535.

4. OK, at the end, I would like to ask you to advice me what I should do next? I don't want EE peoples think badly about me. I will put this question in open status for one more week. After this time, if nobody gives me a solution, I'll put it in delete pending status again.

Thanks to all for help,
VHOA
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2881494
> But no one of them was fully satisfied me.

Then you should explain why. What aspect of the solutions provided would fully satisfy you? What are you looking for but don't know yet?

 > After this time, if nobody gives me a solution,

But, we've already given _two_ solutions!

..B ekim
0
 

Author Comment

by:pvhoa
ID: 2882202
Mike:

I have some comments for your questions:

1. The sentence "...no one of them..." is about the things I have found in EE archive database. This means I haven't found any universal solution for every cases. In general, the answers say about OCR technology (except Anderson Adrian). This technique isn't suitable for me. And Anderson's solution is not good for all too, because almost nowadays complex programs usually use their own private DC to draw. This is noway to force them to draw to our Metafile DC.

2. About your solutions: This is not univeral to, you say the same, right? Because almost programs don't support IAccessibility interface.

Sorry again Mike, and hope that you will get me right.

Best,
VHOA
0
 
LVL 3

Expert Comment

by:V_Bapat
ID: 2890259
> After posting a question, I did some searches on Experts-Exchange trying to find the similar questions and answers. I found a dozen of them. And I spent some hundreds points to get these old answers.

You have not purchased a single question and hence you have not spent a single point. Or at least you have not done that from this user name.
If you have more than one account in Experts Exchange - which is not allowed here - please close other account(s) and keep only one account.
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2891944
> This technique isn't suitable for me

Why not? Coupled with IAccessibilty, it's really the only reasonable way to go about solving your problem.

You've asked how to do something, and we've told you. The normally accepted way to get that work done isn't suitable to you, but you won't tell us why so we can try to find some other mechanism for you.

Instead, you've just deleted your question in an attempt to destroy our work. That's wrong.

..B ekiM
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 

Author Comment

by:pvhoa
ID: 2892704
Hi Mike,

I said OCR isn't a solution for me because I don't know how to program an OCR application. And I will not try to learn about it. It's not for me. One more thing you should recognize that now Window 9x use the antialias technique, which makes the OCR solution so hard.

Mike, I'm not a bad guy. Your help is highly appreciated. Just for fair play, I shared with you a very bright solution of Anderson Adrian, right? OK, if this knowledge sharing isn't enough for you, what do you think about the better for me to do in this case? You and the others shared with me what you know. I share with you what I found. If you think I should share some points for Luisr and yourshelf, I'm ready.

Just let me know about your opinion.

Best,
VHOA

For V_bapat: You are good "hacker" V_Bapat. Who are you? EE Administrator? This is not a place for you to "hack" into. Your comments will be ignored.
0
 

Author Comment

by:pvhoa
ID: 2892738
Hi Mike,

Would you see that we are wasting time like the old ladies? Why don't you recommend me what I should do and we will close the deal?

VHOA
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2894854
> I said OCR isn't a solution for me because I don't know how to program
 > an OCR application.

Then, you need to ask that question. You didn't bother pointing that out until now.

 > And I will not try to learn about it.

That's your choice. But we've still worked to answer the question as you posed it. Just because you've changed your mind doesn't mean our work is worth deletion.

 > shared with you a very bright solution of Anderson Adrian, right?

That solution is of no value to me, and I don't suspect it's of value to the other Experts that went out of their way to help you. And, by your own metrics, it's not acceptable to you because it doesn't work with all applications. You'll need to use a combination of these techniques to get the best coverage you can.

 > Why don't you recommend me what I should do and we will close the deal?

I have, two or three times. I said that you should point out why the solutions offered were unacceptable. If you don't want to write an OCR, then you'll need to use IAccessibility. It'll get you working with many apps, though not all apps that are out there.

You've lured us all in with a huge point total and a "very urgent" demand. Now, it turns out that you're not interested in giving away any ponits--you just want to delete the question and renig on your offer. How could that be urgent?

I guess you can do lots of different things:

1) Nothing. That's bad, I'd say, because it would show that you don't care about Experts, their time, or the help they give.

2) Offer some points to the people who've made suggestions.  From my point of view, this is the right thing to do. And the points aren't all that important (To me, anyway). But it's of prime importance to not having users abuse the system, like you have done, and short-change Experts. That ruins things for everyone, in the long run.

3) Accept one of the many comments here as an answer. There's a part of me that thinks this is what you should do, but I doubt there's a realistic chance of you doing it. You've asked the question, and it's been answered very reasonably--it's a hard problem, and hard problems don't always have complete solutions. (You wouldn't be able to get this code for a thousand _dollars_; expecting something for a thousand useless points is silly.)

Before jumping into EE with both feet, throwing around points and trying to take them back, I think you should have learned how things worked here, first.

..B ekiM
0
 

Author Comment

by:pvhoa
ID: 2895289
Hi Mike,

I said you that I choiced "no-go" for my project that's requires this technique. It's one of the reasons of deleting my question. This is about "Very Urgent" status.

Mike, I think your second advice is best for us. The third is not fair, not because I'm greed regarding points. I know that my question is hard. If it's not so hard, sure that I don't need Experts' advices. And now, I remembered the name of an e-dictionary that I mentioned above. At the same time, I found another one via ZD. Have a look at these site: www.babylon.com and www.gurunet.com

You will see what I called perfect solutions.

One more question that I would like to share 500 points to you and luisr. How can I do it, please tell me.

Best,
VHOA
0
 

Author Comment

by:pvhoa
ID: 2895309
Hi Mike,

I said you that I choiced "no-go" for my project that's requires this technique. It's one of the reasons of deleting my question. This is about "Very Urgent" status.

Mike, I think your second advice is best for us. The third is not fair, not because I'm greed regarding points. I know that my question is hard. If it's not so hard, sure that I don't need Experts' advices. And now, I remembered the name of an e-dictionary that I mentioned above. At the same time, I found another one via ZD. Have a look at these site: www.babylon.com and www.gurunet.com

You will see what I called perfect solutions.

One more question that I would like to share 500 points to you and luisr. How can I do it, please tell me.

Best,
VHOA
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 2929291
pvhoa> I said you that I choiced "no-go" for my project that's requires this technique.
 pvhoa> [...]
 pvhoa> Have a look at these site: www.babylon.com and www.gurunet.com
 pvhoa> You will see what I called perfect solutions.

I installed Babylon.  It's an interesting little program.

But it uses exactly the technique we recommended to you. Prove it to yourself: use ALT+Print Screen to capture a program's image with some text on it. Close that program.
Start Paint. Paste the screenshot you made into paint. Activate Babylon over the bitmap.

Bingo: Since Babylon worked, you know it's using character recognition. There's simply no other way that it could be working because Paint can't represent the data native to the other application. That data is gone!  Paint only has a bitmap.

Using your own metric, the people here have provided you with a perfect solution. It's not our fault the solution you're looking for is arbitrarily not acceptable to you--we've explained exactly how that program works.

 pvhoa> One more question that I would like to share 500 points to you and luisr.

What about yarond?

 pvhoa> How can I do it, please tell me.

Make a posting in the customer service topic. Or, send email to EE's customer service.

..B ekiM
0
 
LVL 10

Expert Comment

by:RONSLOW
ID: 3581852
And still the question remains here ... has anything happened re points to the experts who helped?
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 3617214
Ironic that something so "Urgent!" could languish for three months, huh?

..B ekiM
0
 
LVL 10

Expert Comment

by:RONSLOW
ID: 3628832
mike: I think you should propose an answer .. at least then the auto-grader will (eventually) give you credit for the work you've done and the question will be locked away in the PAQ.  I don't think you'll get any joy out of pvhoa !!!
0
 
LVL 11

Accepted Solution

by:
mikeblas earned 1000 total points
ID: 3629833
I'm afraid you're right.  EE is being ruined by dishonest users.

..B ekiM
0
 
LVL 3

Expert Comment

by:luisr
ID: 3657492
Mmmmm... he's been wanting to share points for so long... just he does not do it... well...
0
 

Author Comment

by:pvhoa
ID: 3692683
Answer accepted
0
 

Author Comment

by:pvhoa
ID: 3692684
Since mike still don't get me right and has shown a great interest in points, I think the best solution is not sharing the points. Better I would accept his excellent answer hoping that he will learn to recognize who is dishonest, and who not here. And please, ronslow, luisr, or any expert else still interest in receiving points from this question just give me a comment. I'm ready to give you another 1300 points that I have now.
0
 
LVL 10

Expert Comment

by:RONSLOW
ID: 3692774
Mike is not particularly interested in points.  But leaving the question around unanswered is not good .. nor is deleting and not awarding those who have givens you the best answers possible (you may not like them, but that doesn't alter the facts).

Implying dishonesty on the part of Mike (unless I am reading your comment incorrectly) is WAY off line .. an apology is definitely in order here.

You wanted a solution .. the only practical one was given and that wasn't good enough for you.

You cited an example of what you wanted instead .. and it used the same methods as the one you didn't like.

You offered to share points .. and over a month later nothing had happened.

I don't believe this puts you in a good position to criticise such experts as Mike.  The experts here have provided you with lots of information and ideas .. certainly enough for you to conclude that you no longer want to attempt your project.  That on its own is worth the points you proposed.

I am sure Mike would have been MORE than happy to share points with the other experts who contributed here (I don't need any as I came here too late to contribute) .. but it is too late now.
0
 
LVL 3

Expert Comment

by:V_Bapat
ID: 3751412
Finally.....
0
 
LVL 11

Expert Comment

by:mikeblas
ID: 3752480
> I think the best solution is not sharing the points. Better
 > I would accept his excellent answer hoping that he will learn
 > to recognize who is dishonest, and who not here.

Why? If you wanted to share the points, why didn't you? You could've done it last month.  Or, you could've still done it before accepting my answer.  From the looks of it to me (and, apparently, to RONSLOW, too) you weren't going to be offering _any_ points. You let the question languish--and even tried to delete it!

The ball was in your court the whole time. For months. Blaming me for your problems is absolutely absurd.

If the other experts who contributed want to receive points for there help here, I'll be happy to open questions _of my own_, using _my own points_, and pass 'em out. Just let me know by posting a comment, here.

Showing a great interest in points is only natural. It's the only thing that Experts can look forward to! Waving around 1000 points, getting a bunch of advice, and then trying to retract the points is despicable. It's tedious to recover from a user who doesn't grade their questions and apparently didn't really mean to offer any points in the first place, but I'm sure we can make it through.

..B ekiM
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Introduction: Load and Save to file, Document-View interaction inside the SDI. Continuing from the second article about sudoku.   Open the project in visual studio. From the class view select CSudokuDoc and double click to open the header …
Introduction: Dialogs (1) modal - maintaining the database. Continuing from the ninth article about sudoku.   You might have heard of modal and modeless dialogs.  Here with this Sudoku application will we use one of each type: a modal dialog …
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now