Solved

Sending Mouse Move, Mouse Clicks, Keyboard events to another application

Posted on 2004-04-07
9
3,119 Views
Last Modified: 2009-08-05
What I'm trying to accomplish:  Given a class name and windows name I want
to simulate keyboard and mouse events on that window.

For instance if notepad is is started the default window name is "Untitled -
Notepad" class name is "Notepad".
I want to simulate the user typing "hello there" press enter key fo new
line.  Then type "bob".  Then position the mouse over the close button and
simulate a mouse click.

That may be a dumb example but basically I'm in the process of creating a
program that can do macros on another program.  By macros again I mean
simulating that the user behaved in such a way as though it appeared that
the user actually was hitting the keyboard buttons moving the mouse and
clicking with the mouse.

The first thing I do is enumerate through all the windows and store the
window handle, the process id, the thread id that created the window, the
handle of the thread that created the window (haven't figured this out yet),
the window name, the class name.

After the user determines the window I want them to be able to run a macro.
Right now I haven't gotten far enough along to actually save stuff.  My
macro is just typing hello.  The typing of hello should send a key down of
h, a key up of h, a key down of e, a key up of e, etc.  I have this working
as well.

I'd also like to simulate a mouse click on a certain point.  This I do not
have working.  It should simulate a mouse move to a particular x, y
location.  Then left mouse click down and finally a left mouse click up.

Again I have all of this working except for the mouse clicking.

Here is the code that I use to type "hello" in notepad, press the enter key,
and then left click on the screen at (468, 580).  Everything works except
for the left click which seems to always be clicking at position 0, 0 on the
screen.  I will define the Inputs class in just a moment.

  if(AttachThreadInput(GetCurrentThreadId(), threadID, true))
  {
    ShowWindow(hWnd, SW_MAXIMIZE);
    ShowWindow(hWnd, SW_SHOWDEFAULT);

    Inputs inputs;
    inputs.writeString("hello");
    inputs.type(VK_RETURN);
    inputs.leftClick(468, 580);

    inputs.sendInputs();

    AttachThreadInput(GetCurrentThreadId(), threadID, false);
  }

Ok this code is relatively simple.  Only thing I don't like is that if the
application is minimized I have to do ShowWindow(hWnd, SW_MAXIMIZE);
ShowWindow(hWnd, SW_SHOWDEFAULT);
In order to get the input to be received by the program.  If the window is
already being displayed those two lines aren't needed.

the line that says
inputs.sendInputs();
All it does is to use the windows api function SendInput which takes
arguments of number of inputs, pointer to array of INPUT structures, and the
size of the INPUT structure.  It returns the number of inputs processed.

To ease my handling of all the windows api function calls I created a class
called Inputs which simply adds an INPUT structure to a list for each event
that is required.  For instance Inputs.type(VK_RETURN) adds two INPUT
structures to my list.  Both are of type KEYBOARDINPUT.  One simulates a the
enter key being pressed...the other simulates it being released.

I will provide the entire Inputs class.  Again all is working except for the
mouse clicks and think the problem actually lies in the mouseMove(int x, int
y) method as it always seems to move the mouse to screen coordinates 0, 0

//******************** Inputs.h ****************************
//--------------------------------------------------------------------------
-
#ifndef InputsH
#define InputsH


#include <string>
#include <list>
#include <windows.h>

//--------------------------------------------------------------------------
-
class Inputs
{
public:
  Inputs();
  ~Inputs();

  void clear();

  void keyDown(WORD wVk);
  void keyUp(WORD mVk);
  void typeChar(char ch);
  void type(WORD mVk);

  void writeString(std::string str);

  void leftClickDown();
  void leftClickUp();
  void leftClick();

  void rightClickDown();
  void rightClickUp();
  void rightClick();

  void leftClickDown(int x, int y);
  void leftClickUp(int x, int y);
  void leftClick(int x, int y);

  void rightClickDown(int x, int y);
  void rightClickUp(int x, int y);
  void rightClick(int x, int y);

  void moveMouse(int x, int y);

  unsigned sendInputs();
private:
  PINPUT createInputsPtr();
  typedef std::list<INPUT> INPUT_LIST;
  INPUT_LIST inputs;
  unsigned numInputs;
};
//--------------------------------------------------------------------------
-
#endif
//--------------------------------------------------------------------------
-


//******************** Inputs.cpp ***************************
//--------------------------------------------------------------------------
-
#pragma hdrstop

#include "Inputs.h"
//--------------------------------------------------------------------------
-
#pragma package(smart_init)
//--------------------------------------------------------------------------
-

Inputs::Inputs()
{
  clear();
}

Inputs::~Inputs()
{
}

void Inputs::clear()
{
  inputs.clear();
  numInputs = 0;
}

void Inputs::keyDown(WORD wVk)
{
  INPUT input;
  input.ki.wVk = wVk;
  input.ki.wScan = 0;
  input.ki.dwFlags = 0; //press down;
  input.ki.time = 0;
  input.ki.dwExtraInfo = 0;

  input.type = INPUT_KEYBOARD;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::keyUp(WORD mVk)
{
  INPUT input;
  input.ki.wVk = mVk;
  input.ki.wScan = 0;
  input.ki.dwFlags = KEYEVENTF_KEYUP;
  input.ki.time = 0;
  input.ki.dwExtraInfo = 0;

  input.type = INPUT_KEYBOARD;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::typeChar(char ch)
{
  if(ch >= 'A' && ch <= 'Z')
  {
    keyDown(VK_SHIFT);
    type(ch);
    keyUp(VK_SHIFT);
  }
  else if(ch >= 'a' && ch <= 'z')
  {
    type(ch - 'a' + 'A');
  }
  else
  {
    type(ch);
  }
}

void Inputs::type(WORD mVk)
{
  keyDown(mVk);
  keyUp(mVk);
}


void Inputs::writeString(std::string str)
{
  const char* ptr = str.c_str();
  unsigned length = str.length();

  for(unsigned i = 0; i < length; i++)
    typeChar(ptr[i]);
}

void Inputs::leftClickDown()
{
  INPUT input;
  input.mi.dx = 0;
  input.mi.dy = 0;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::leftClickUp()
{
  INPUT input;
  input.mi.dx = 0;
  input.mi.dy = 0;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_LEFTUP;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::leftClick()
{
  leftClickDown();
  leftClickUp();
}

void Inputs::rightClickDown()
{
  INPUT input;
  input.mi.dx = 0;
  input.mi.dy = 0;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_RIGHTDOWN;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::rightClickUp()
{
  INPUT input;
  input.mi.dx = 0;
  input.mi.dy = 0;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_RIGHTUP;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

void Inputs::rightClick()
{
  rightClickDown();
  rightClickUp();
}

void Inputs::leftClickDown(int x, int y)
{
  moveMouse(x, y);
  leftClickDown();
}

void Inputs::leftClickUp(int x, int y)
{
  moveMouse(x, y);
  leftClickUp();
}

void Inputs::leftClick(int x, int y)
{
  moveMouse(x, y);
  leftClickDown();
  leftClickUp();
}

void Inputs::rightClickDown(int x, int y)
{
  moveMouse(x, y);
  rightClickDown();
}

void Inputs::rightClickUp(int x, int y)
{
  moveMouse(x, y);
  rightClickUp();

}

void Inputs::rightClick(int x, int y)
{
  moveMouse(x, y);
  rightClickDown();
  rightClickUp();
}

void Inputs::moveMouse(int x, int y)
{
  INPUT input;
  input.mi.dx = x;
  input.mi.dy = y;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

unsigned Inputs::sendInputs()
{
  unsigned result;

  PINPUT pInputs = createInputsPtr();
  result = SendInput(numInputs, pInputs, sizeof(INPUT));
  delete[] pInputs;

  return result;
}

PINPUT Inputs::createInputsPtr()
{
  PINPUT pInputs = new INPUT[numInputs];
  INPUT_LIST::iterator itr = inputs.begin();
  unsigned i = 0;
  while(itr != inputs.end())
  {
    pInputs[i++] = (*itr);
    itr++;
  }
  return pInputs;
}
0
Comment
Question by:PerryDK
9 Comments
 
LVL 10

Expert Comment

by:Sys_Prog
ID: 10781545
PerryDK,

Did u debug ur code, and what is the return value for the succesfully executed inputs

Could u post ur complete code

Amit
0
 
LVL 16

Expert Comment

by:nonubik
ID: 10781669
Yeah, the SendInput(..) method would clarify the problem. To see how you send leftclick.
0
 
LVL 4

Expert Comment

by:caner_elci
ID: 10782354
PerryDK,

I suggest you to check SetWindowsHookEx() function and especially WH_JOURNALPLAYBACK hook. I don't know how do you record macros, but you can use WH_JOURNALRECORD hook for it. I think JournalPlaybackProc would work fine for you. In fact, it might work better than SendInput() function.. Just give it a try, and if you need any help about it, just let me know..

Caner
0
 
LVL 4

Author Comment

by:PerryDK
ID: 10785549
SendInput  refer to http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/WinUI/WindowsUserInterface/UserInput/KeyboardInput/KeyboardInputReference/KeyboardInputFunctions/SendInput.asp

It is a windows function.

That is all the code you should need unless you want to see the really nitty-gritty stuff like enumerating through all the windows to find the right class name and window name to retrieve the Handle to the window. And the threadID that created the window.

The problem lies in this method
void Inputs::moveMouse(int x, int y)
{
  INPUT input;
  input.mi.dx = x;
  input.mi.dy = y;
  input.mi.mouseData = 0;
  input.mi.dwFlags = MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE;
  input.mi.time = 0;
  input.mi.dwExtraInfo = 0;

  input.type = INPUT_MOUSE;

  inputs.push_back(input);
  numInputs++;
}

refer to this http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/WinUI/WindowsUserInterface/UserInput/KeyboardInput/KeyboardInputReference/KeyboardInputStructures/MOUSEINPUT.asp
 for info on the MOUSEINPUT structure.

The x and y I'm passing in to this function which range from (0,0) to (800, 600) must not be in screen coordinates as it always moves the mouse to the upper left of the screen.  This is the function that I really need help with.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 4

Accepted Solution

by:
caner_elci earned 500 total points
ID: 10791004
PerryDK,

Okay, just replace this code with yours and you'll pain will blow away.. :)

input.mi.dx = x*65535/GetSystemMetrics(SM_CXSCREEN);
input.mi.dy = y*65535/GetSystemMetrics(SM_CYSCREEN);

Use this conversion while calculating x and y coordinates..

Hope it will help,

Caner ELCI
0
 
LVL 4

Expert Comment

by:caner_elci
ID: 10791028
some typo, your pain was supposed to blow away, not you :)
0
 
LVL 4

Author Comment

by:PerryDK
ID: 10793291
Thanks Caner!!!

I'll accept your answer because it definitly works.  But can you provide a brief description of whats going on.  It looks as though I'm converting an (x, y) in some unknown coordinate sytem to an (x, y) in screen coordinates.  What coordinate system I'm a converting from?
0
 
LVL 4

Expert Comment

by:caner_elci
ID: 10794707
Hi PerryDK,

Since SendInput() function requires a value between 0 and 65535 for x and y coordinates, you should do that conversion to find the position scaled to primary display's resolution. That's why we multiply coordinates with 65535 and divide into screen width and height.
0
 
LVL 4

Author Comment

by:PerryDK
ID: 10807454
Sorry it took me so long to accept answer was visiting family for easter.  Great answer!
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Errors will happen. It is a fact of life for the programmer. How and when errors are detected have a great impact on quality and cost of a product. It is better to detect errors at compile time, when possible and practical. Errors that make their wa…
Introduction This article is a continuation of the C/C++ Visual Studio Express debugger series. Part 1 provided a quick start guide in using the debugger. Part 2 focused on additional topics in breakpoints. As your assignments become a little more …
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now