?
Solved

Reading the function from a function pointer

Posted on 2008-11-14
21
Medium Priority
?
387 Views
Last Modified: 2012-05-05
Background information below, if you don't know anything about genetic programming, you can probably just skip this:
I interface with a Genetic Programming-program called Discipulus.
I load a DLL into Discipulus that does the necessary calculations to return a fitness value on each individual.
For each individual Discipulus runs my function below.
I disagree on discipulus on which GP-programs that are interesting enough to be saved and examined further, which is why I'm gonna try to add an extra "layer" of saving functionality

Question:
I want to attempt reading the content of the function pointer directly. I get a pointer and the length of the program at that location. How would I get this data out to a textfile so I can try to make some sense out of it?
typedef void (* FUNCTION_PTR) (float*, float**);
 
float CalculateFitnessError(FUNCTION_PTR pGeneticProgram, long Length) {
    //Load some inputs
    (*pGeneticProgram)(Inputs,Outputs); //Run the GP-program
    //Do some math on the outputs
    //Return a fitness value
}
 
I imagine that the answer to my question, in pseudo code, would look something like:
 
float CalculateFitnessError(FUNCTION_PTR pGeneticProgram, long Length) {
    //Load some inputs
    (*pGeneticProgram)(Inputs,Outputs); //Run the GP-program
    //Do some math on the outputs
    If(FitnessValue < Threshold) {
        var GPP = read Length bytes from adress pGeneticProgram
        save GPP to someFile
    }
    //Return a fitness value
}

Open in new window

0
Comment
Question by:letharion
  • 11
  • 10
21 Comments
 
LVL 53

Expert Comment

by:Infinity08
ID: 22958489
You want to reverse-engineer the code of a function in the Discipulus code ?
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22958501
I don't get what you're trying to do.

>> read Length bytes from adress pGeneticProgram

pGeneticProgram is a function pointer - it points to the code of that function (its implementation). It does not point to any data you can analyze.
0
 
LVL 6

Author Comment

by:letharion
ID: 22959341
"pGeneticProgram is a function pointer - it points to the code of that function (its implementation). It does not point to any data you can analyze."
Yes, and no. I understand (I think) that it points to an implementation of a function.
I also understand that I cannot "analyse" the data.

But when I tell Discipulus that I think it's finished searching for functions, I'm not statisfied with the list of "best functions", since discipilus lists them in order of "best on training data", and I'm more interested in how they performed outside of the training set.

Therefore, I'd like to figure out a way to copy the function directly when I deem it "good enough". And since I do have "complete" access to the function, in that I have get the pointer, I figured this should be doable. But maybe I'm making erroneous assumptions somewhere along the way. :)
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 53

Expert Comment

by:Infinity08
ID: 22959443
>> I'd like to figure out a way to copy the function directly when I deem it "good enough". And since I do have "complete" access to the function, in that I have get the pointer

If you have a pointer to the function, you can call the function with any parameters you want. Just like in the first part of your example code.


If the length parameter is the actual size of the function in memory, you can dump that function to a (binary) file if you want, and then try to reverse-engineer it. A debugger will help with that by dumping the assembly instructions for the function.

I'm not sure why though. Isn't the "goodness" of the function determined by running it on test data rather than analyzing its code manually ?
0
 
LVL 6

Author Comment

by:letharion
ID: 22959961
The length parameter is according to my specifications "The length of the program" that the first paramter points to. I hope that will be enough.

"You can dump that function to a (binary) file if you want, and then try to reverse-engineer it. A debugger will help with that by dumping the assembly instructions for the function."
That would be precisely what I would like to do :)
However, you mention using a debugger (as in  a separate program). Is the process complicated enough to need that? I would have thought/hoped that I could accomplish this "easily" as in, say less than 100 lines of code? Have I completely underestimated the complexity of the problem? (Just dumping the function to a file that is)

On the subject of "why". You're absolutely correct that the "goodness" is determined that way, but after say a run of 1 hour, many thousands of programs have tested, and automatically, the GP only saves a handful of the very best of them for me. I know by testing on both training and non-training data that there are a lot of those programs that could perform mediocerly on both training and non-training data.
I'm interested in having a closer look on these programs.
0
 
LVL 6

Author Comment

by:letharion
ID: 22959983
Clarification:
I'd rather get a program that performs medicerly on both data sets, than one overfitted on one of them, and not working on the other.
Have I made any sense? ;-)
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22960096
>> Is the process complicated enough to need that?

Yes. The data that will be dumped to the file will be a simpled memory dump of the function. It will be object code (generated by a compiler) ready to be run by the processor. It corresponds directly to assembler instructions, so a debugger can easily show you the assembler instructions for the function.


>> Have I completely underestimated the complexity of the problem? (Just dumping the function to a file that is)

You will not get simple C code as you were maybe thinking. So, I think it's indeed a bit more involved than you imagine.


>> the GP only saves a handful of the very best of them for me.

I see, and you want to test them all - even the ones Discipulus considered not good enough.
I think the easiest way would be to have your callback function run some non-training data through the function to see the results.


That is, if I understood correctly what Discipulus does. Correct me if I'm wrong, but here's what I imagine it does : it generates several functions, and for each of those functions, it calls your callback function (CalculateFitnessError) at least once. Based on the result of your callback function, it decides whether the generated function should be kept or not.
Is that correct ? If not, can you explain how it works ? (I'm not familiar with Discipulus, but I am familiar with genetic programming)
0
 
LVL 6

Author Comment

by:letharion
ID: 22960539
First, thanks for helping me Infinity :)

You have understood what Discipulus does perfectly.

However, as you suggest, I do already test on some non-training data to see how the function fares outside of it's training. And when I think that result looks interesting, I'd like to copy it so that I can examine it further manually.

I do realise that I will unfortunately not get C code when I do a memory dump.

Ok, now that we're this far, are there any practical suggestions you can give me on how to take the next step? Even using a debugger, should I still dump the memory to a file and then feed it to a debugger? Or should I invoke the debugger directly from my callback function? What debugger do you recommend?
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22960628
>> I'd like to copy it so that I can examine it further manually.

Ok.

Alternatively, you could "trick" Discipulus into keeping the function by returning a "good" output, even though the training data result might have been bad. Is that an option ?


>> Even using a debugger, should I still dump the memory to a file and then feed it to a debugger?

That would be easiest, since then you can do it offline, and you don't depend on Discipulus for keeping the functions in memory.


>> Or should I invoke the debugger directly from my callback function?

Instead of dumping the memory dump to the file, you could first use the debugger to output the assembler, and then dump that to a file (or otherwise handle it). But it's more straightforward to just dump the memory to file, and process it later, manually.


>> What debugger do you recommend?

That depends on your platform, OS and compiler you used.
0
 
LVL 6

Author Comment

by:letharion
ID: 22960747
>Alternatively, you could "trick" Discipulus into keeping the function by returning a "good" output, even >though the training data result might have been bad. Is that an option?
Hmmm, perhaps it could be, I'll think about it, but I think I'd prefer not to, as this could potentially have consequences for it's continued searching


>> Even using a debugger, should I still dump the memory to a file and then feed it to a debugger?
>That would be easiest, since then you can do it offline, and you don't depend on Discipulus for keeping >the functions in memory.
Any chance you could point to me to where I could learn to do just that? I don't know what syntax I would use to read memory.

>> What debugger do you recommend?
>That depends on your platform, OS and compiler you used.
Wintel XP, Microsoft Visual Studio 2008
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22964484
>> as this could potentially have consequences for it's continued searching

True.


>> Any chance you could point to me to where I could learn to do just that? I don't know what syntax I would use to read memory.

You can use the function pointer as if it were pointing to an array of bytes of a given length. You can then simply write those bytes to a file using fwrite :

        http://www.cplusplus.com/reference/clibrary/cstdio/fwrite.html


>> Wintel XP, Microsoft Visual Studio 2008

I don't know Visual Studio too well, so I can't say whether it has the capability to do this (I would assume so - maybe ask someone who knows it better than me).
But the obvious alternative is a disassembler for Windows. Common ones are IDA Pro (http://www.hex-rays.com/idapro/) and nasm (http://sourceforge.net/projects/nasm).
0
 
LVL 6

Author Comment

by:letharion
ID: 22967619
>You can use the function pointer as if it were pointing to an array of bytes of a given length.
I have gotten myself confused over the syntax of function pointers, mainly because I'm not used to programming with pointers at all.
But I thought, I'd try what you said.

So I take a pointer to an array of bytes and try accessing them.
The code in "Sample 1" compiles. (But doesn't run, supposedly because I haven't allocated or initialized the memory)

However, the way I understand what you are saying, I should then be able to apply the same syntax to the function pointer. Sample 2 doesn't compile because "subscript requires array or pointer type" which if I understand it that I can't do [] on *pGeneticProgram because pGeneticProgram isn't a pointer or an array.
Sample 1:
byte* a[10];
cout << *a[0] << *a[1];
 
Sample 2:
cout << *pGeneticProgram[0] << *pGeneticProgram[1];

Open in new window

0
 
LVL 6

Author Comment

by:letharion
ID: 22967635
I experimented with the syntax a bit, trying
cout << pGeneticProgram[0]
cout << &pGeneticProgram
cout << *pGeneticProgram
cout << **pGeneticProgram
cout << *(*pGeneticProgram)
and the like, and for the versions that do compile, I can get some hex-numbers.
Since I don't understand however what I'm doing, I'm not sure if I'm looking at a memory adress , or if it's assembly instructions simply interpreted as ascii letters.
0
 
LVL 53

Accepted Solution

by:
Infinity08 earned 2000 total points
ID: 22968294
>> byte* a[10];

is an array of 10 pointers to byte.

A pointer to one or more bytes is simply :

        byte *a;

The first byte can be accessed using a[0], the second byte as a[1], etc.

In your case, you have a function pointer, and you want to write the whole function into a file :
float CalculateFitnessError(FUNCTION_PTR pGeneticProgram, long Length) {
    FILE *out = fopen("out.bin", "wb");
    fwrite(pGeneticProgram, 1, Length, out);
    fclose(out);
}

Open in new window

0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22968295
Of course the CalculateFitnessError function also has to do its normal job and return a float.
0
 
LVL 6

Author Closing Comment

by:letharion
ID: 31516726
Big thanks Infinity :)
0
 
LVL 6

Author Comment

by:letharion
ID: 22975184
Bah, stupid me for accepting a solution before trying it... I'm just gonna hope you follow this still Infinity.

Your code only produces a 0-byte file.

The code I tried replacing it with gives me "01CBA880" everytime, even after restaring discipulus.
Any idea what/why that is?
//Below code produces 0 byte empty file.
FILE *out = fopen("out.bin", "wb");
fwrite(pGeneticProgram, 1, Length, out);
fclose(out);
 
//Tried replacing with this
ofstream logfile;
logfile.open("c:\\file.txt", ios::app);
logfile.close();

Open in new window

0
 
LVL 6

Author Comment

by:letharion
ID: 22975187
Maybe, just maybe I should have included the line where I actually send data to the logfile ^^
//Tried replacing with this
ofstream logfile;
logfile.open("c:\\file.txt", ios::app);
logfile << pGeneticProgram << endl << endl;
logfile.close();

Open in new window

0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22975335
>> Your code only produces a 0-byte file.

Are you sure that Length has a value different from 0 ?


>> The code I tried replacing it with gives me "01CBA880" everytime

Your code writes the pointer to the file, not the data the pointer is pointing to.
If you want to use C++ streams, then use this code :
std::ofstream out("out.bin", std::ofstream::binary);
out.write(pGeneticProgram, Length);
out.close();

Open in new window

0
 
LVL 6

Author Comment

by:letharion
ID: 22984072
*Sigh* Thanks infinity, the Length often (but not always) set to 0, even when the program isn't of 0 length. I've mailed the makers of Discipulus about it.

I'm starting to drift here, so maybe I should open another question,
but is there a "signature" to the end of  functions that I could look for?

Like I could find the end of a string by looking for it's terminating '\0'.

That way I wouldn't have to depend on the Length variable at all.
0
 
LVL 53

Expert Comment

by:Infinity08
ID: 22984100
>> but is there a "signature" to the end of  functions that I could look for?

Every compiler has its own peculiarities, and then there are different compiler options and optimization levels that make things even more complicated.
What's worse, is that in this case, the code is generated by Discipulus.
So, there's no generic way to identify the end of a function.

You could always dump a reasonable amount (larger than the largest possible function), and then analyze it.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Unlike C#, C++ doesn't have native support for sealing classes (so they cannot be sub-classed). At the cost of a virtual base class pointer it is possible to implement a pseudo sealing mechanism The trick is to virtually inherit from a base class…
What is C++ STL?: STL stands for Standard Template Library and is a part of standard C++ libraries. It contains many useful data structures (containers) and algorithms, which can spare you a lot of the time. Today we will look at the STL Vector. …
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.
Suggested Courses

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question