How to process returned pointer structure data from Windows DLL in Ruby

In Ruby, Call or invoke a API DLL library is easily via Win32API class, win32-api gem or other gems. For general DLL API call, there are quite a few references, some good tips list below:
http://www.rubytips.org/2008/05/13/accessing-windows-api-from-ruby-using-win32api-library/
http://www.ruby-doc.org/docs/ProgrammingRuby/html/lib_windows.html

But all of these articles, tips don't mention how to process the returned pointer of a struct or memory block. As we known, standard DLL routine should return a code and return pointer/struct/memory block by function arguments, memory should free by claimer, so we can allocate memory in our codes and pass this pointer into DLL, the DLL routine save returned data into that pointer, after processed we need to free the allocated pointer ourselves. Anyway, the DLL author might not follow the General Programming Rules, or have some special reason, the DLL maybe return a pointer structure data directly instead of function arguments.

For example, if a definition like below:

typedef  struct Data {
                          int len;
                          int flag;
                          char values[1000];
                        } *Data;
                        PData = *Data;

Select all Open in new window

and the exports function of the DLL like below:

Data* GetData(int index, int flag); cdcel;

Select all Open in new window

Win32API only provider a few data types like "0", "V", "I", "L", "P", other data types of other languages will mapping to these types, then we using "P" to receive the pointer data. unfortunately, we can't get the correct data by Win32API in Ruby via normally way:

require "Win32API"
                      getData = Win32API.new('filename.dll', 'GetData', ['I', 'I'], 'P');
                      pdata = getData.call(100, 10)

Select all Open in new window

We might not get the correct data by pdata, because the pdata might be truncated. The "P" declaration only worked for import sections but does not work as well for export section.

In fact, Ruby doesn't know the size of the return pointer's object, thus Ruby just process pointer's data like "null-terminated" string. so the returned data might be truncated if the returned pointer's object is a complex struct. A similar question appears here demonstrating this problem: http://www.iteye.com/topic/47936

We hope Ruby will improve this later, but how do we process the PData pointer after the call now?

If we want to get returned data correctly, first, we need to define another API first:

@CopyMemory = Win32API.new('Kernel32.dll', 'RtlMoveMemory', ['P', 'I', 'I'], '0')

Select all Open in new window

then, we need to change the Win32API export of the API as 'L':

getData = Win32API.new('filename.dll', 'GetData', ['I', 'I'], 'L')

Select all Open in new window

now, we can call the API like before:

pd = getData.call(100, 10)

Select all Open in new window

Last, we get data by the pointer via CopyMemory API:

buf = Array.new(buf_size, 0).pack('L*')
                      @CopyMemory.call(buf, pd, buf_size)

Select all Open in new window

The buf saved the returned data now. you can process them, for this tips, we can process the data like below:

len, flag = buf.unpack('I2')
                      values = buf[9, len]

Select all Open in new window

Of course, you should give the right buf_size, otherwise you might get AV error or crash.