How to process returned pointer structure data from Windows DLL in Ruby

Published on
10,364 Points
Last Modified:
In Ruby, Call or invoke a API DLL library is easily via Win32API class, win32-api gem or other gems. For general DLL API call, there are quite a few references, some good tips list below:

But all of these articles, tips don't mention how to process the returned pointer of a struct or memory block. As we known, standard DLL routine should return a code and return pointer/struct/memory block by function arguments, memory should free by claimer, so we can allocate memory in our codes and pass this pointer into DLL, the DLL routine save returned data into that pointer, after processed we need to free the allocated pointer ourselves. Anyway, the DLL author might not follow the General Programming Rules, or have some special reason, the DLL maybe return a pointer structure data directly instead of function arguments.

For example, if a definition like below:
typedef  struct Data {
    int len;
    int flag;
    char values[1000];
  } *Data;
  PData = *Data;

Open in new window

and the exports function of the DLL like below:
Data* GetData(int index, int flag); cdcel;

Open in new window

Win32API only provider a few data types like "0", "V", "I", "L", "P", other data types of other languages will mapping to these types, then we using "P" to receive the pointer data. unfortunately, we can't get the correct data by Win32API in Ruby via normally way:
require "Win32API"
getData = Win32API.new('filename.dll', 'GetData', ['I', 'I'], 'P');
pdata = getData.call(100, 10)

Open in new window

We might not get the correct data by pdata, because the pdata might be truncated. The "P" declaration only worked for import sections but does not work as well for export section.

In fact, Ruby doesn't know the size of the return pointer's object, thus Ruby just process pointer's data like "null-terminated" string. so the returned data might be truncated if the returned pointer's object is a complex struct. A similar question appears here demonstrating this problem: http://www.iteye.com/topic/47936

We hope Ruby will improve this later, but how do we process the PData pointer after the call now?

If we want to get returned data correctly, first, we need to define another API first:
@CopyMemory = Win32API.new('Kernel32.dll', 'RtlMoveMemory', ['P', 'I', 'I'], '0')

Open in new window

then, we need to change the Win32API export of the API as 'L':
getData = Win32API.new('filename.dll', 'GetData', ['I', 'I'], 'L')

Open in new window

now, we can call the API like before:
pd = getData.call(100, 10)

Open in new window

Last, we get data by the pointer via CopyMemory API:
buf = Array.new(buf_size, 0).pack('L*')
@CopyMemory.call(buf, pd, buf_size)

Open in new window

The buf saved the returned data now. you can process them, for this tips, we can process the data like below:
len, flag = buf.unpack('I2')
values = buf[9, len]

Open in new window

Of course, you should give the right buf_size, otherwise you might get AV error or crash.

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Join & Write a Comment

A query can call a function, and a function can call Excel, even though we are in Access. This is Part 2, and steps you through the VBA that "wraps" Excel functionality so we can use its worksheet functions in Access. The declaration statement de…
Check How effective MS Exchange Expert thinks Exchange Mailbox Recovery by SysTools IS. Visit the Official site to get detailed information:- https://www.systoolsgroup.com/exchange-recovery.html (https://www.systoolsgroup.com/exchange-recovery.h…

Keep in touch with Experts Exchange

Tech news and trends delivered to your inbox every month