Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.
structure packing between compilers. Are they commited to anything ?
Hi!
I am writing a DLL (C++). The DLL has a C API (there is a nice header file, which contains the structures).
Part of the API is structures to be passed In and Out.
Now, the users of my DLL may use many compilers (intel, gcc & microsoft - and also different version of the compilres).
How do I make sure that they all understand my structures "the same way" ?
Suppose I write the following line at the beggining of the header file:
#pragma pack(X) // where x may be 1,2,4 or 8
and ten I write the following line at the end of the header file:
#pragma pack()
Will this do the job in 100% of the cases ?
Thanks zmau
I am writing a DLL (C++). The DLL has a C API (there is a nice header file, which contains the structures).
Part of the API is structures to be passed In and Out.
Now, the users of my DLL may use many compilers (intel, gcc & microsoft - and also different version of the compilres).
How do I make sure that they all understand my structures "the same way" ?
Suppose I write the following line at the beggining of the header file:
#pragma pack(X) // where x may be 1,2,4 or 8
and ten I write the following line at the end of the header file:
#pragma pack()
Will this do the job in 100% of the cases ?
Thanks zmau
Asked:
6-
4
3- +2
2 Solutions
I assume that you don't have endian issues across the platforms. Some pragma pack had a third parameter to handle this (which gcc ignores).
I assume that the struct are c-like POD struct.
I assume that you are not dereferencing pointers to data fields in the struct. That might cause problems if not especially careful.
I assume that you are not concerned about any performance penalties using pragma pack
Even with these assumptions, pragma pack is not in the C or C++ standard. I would not use it for that reason across multiple platforms or compilers.
I assume that the struct are c-like POD struct.
I assume that you are not dereferencing pointers to data fields in the struct. That might cause problems if not especially careful.
I assume that you are not concerned about any performance penalties using pragma pack
Even with these assumptions, pragma pack is not in the C or C++ standard. I would not use it for that reason across multiple platforms or compilers.
phoffric, your assuptions are correct.
BTW, Intel, gcc and Microsoft compiler do "accept pragma pack".
one more issue :
Do these compilers "re-arrage" the data in the structure (i.e. put the last declared variable at the beggining address (for example)) ?
Thanks
BTW, Intel, gcc and Microsoft compiler do "accept pragma pack".
one more issue :
Do these compilers "re-arrage" the data in the structure (i.e. put the last declared variable at the beggining address (for example)) ?
Thanks
>> So, what is the best way to handle this problem ?
I suppose that is another question that will elicit multiple opinions on the "best" way. I'll just jot down some opinions without claiming them to be the "best".
You may be interested in reading this link: DLL_Hell
http://en.wikipedia.org/wiki/DLL_Hell
I suppose that is another question that will elicit multiple opinions on the "best" way. I'll just jot down some opinions without claiming them to be the "best".
You could compile the DLL code for each target to generate a suitable dll.
You could textify (or use xml) the struct into a single text string.
You may be interested in reading this link: DLL_Hell
http://en.wikipedia.org/wiki/DLL_Hell
>> one more issue :
>> Do these compilers "re-arrage" the data in the structure (i.e. put the last declared variable at the beggining address (for example)) ?
I suppose that is another question also.
In my experience when debugging, I have never seen a rearrangement of struct members; and, I have never seen code which reliies on this experience. Experience in such matters in not important. There is nothing in the C++ standard which talks about the layout of POD-struct members. The access to these are through standard operators such as . and ->. In case you were thinking of trying some offset scheme, I would not suggest that if trying to be portable.
>> Do these compilers "re-arrage" the data in the structure (i.e. put the last declared variable at the beggining address (for example)) ?
I suppose that is another question also.
In my experience when debugging, I have never seen a rearrangement of struct members; and, I have never seen code which reliies on this experience. Experience in such matters in not important. There is nothing in the C++ standard which talks about the layout of POD-struct members. The access to these are through standard operators such as . and ->. In case you were thinking of trying some offset scheme, I would not suggest that if trying to be portable.
you could add a member to your structures which holds the size of the structure and must be set from sender. then the receiver can check for the size before accepting the buffer. that also is a check on expected alignment cause different alignment would result into different size.
generally each compiler has an option for packed alignment and as you need individual build tools anyhow, the setting for the packing shouldn't add less portability. i wouldn't set the alignment via pragma preprocessor statement but would use project or makefile options.
Sara
generally each compiler has an option for packed alignment and as you need individual build tools anyhow, the setting for the packing shouldn't add less portability. i wouldn't set the alignment via pragma preprocessor statement but would use project or makefile options.
Sara
I can't figure out why you care about how the struct members are laid out.
Are you using some kind of direct (zero-effort) conversion between structs and binary blobs used in interchange? This is generally a bad idea. Parse the interchange data into a struct, then serialize the struct back into a blob. Very little effort is required.
Are you creating data structures that you want your API users to directly reference? This is a bad idea. What happens when you need to change the struct or add something to it? It creates an unnecessarily intimate dependency between your dll and your users. Instead have the users access the data structure through functions, so the details of the structure are hidden from them. Encapsulation is your friend.
Are you using some kind of direct (zero-effort) conversion between structs and binary blobs used in interchange? This is generally a bad idea. Parse the interchange data into a struct, then serialize the struct back into a blob. Very little effort is required.
Are you creating data structures that you want your API users to directly reference? This is a bad idea. What happens when you need to change the struct or add something to it? It creates an unnecessarily intimate dependency between your dll and your users. Instead have the users access the data structure through functions, so the details of the structure are hidden from them. Encapsulation is your friend.
I m sorry that my question was not clear.
The DLL alraedy exists. It's functions have structures as part of the API.
For example :
typedef struct BlaBla
{
int A:
int B;
} BLA_BLA;
void func_1(BLA_BLA* ptr);
The question is what to do now ?
The DLL alraedy exists. It's functions have structures as part of the API.
For example :
typedef struct BlaBla
{
int A:
int B;
} BLA_BLA;
void func_1(BLA_BLA* ptr);
The question is what to do now ?
>> I am writing a DLL
This is very clear. You have source code and are building a DLL, right?
So, just compile the source on each platform as suggested.
This is very clear. You have source code and are building a DLL, right?
So, just compile the source on each platform as suggested.
Even if the members A and B fall nicely into place in your tests on all platforms, do you expect that the executable statements in the binary DLL will work from one system to another?
for a structure with two int members you don't need to expect alignment issues (but endianess can play a role). different alignment happens when there are members of different size or array members with odd array lengths. for example a char[3] member and a short mixed with int members likely will give different aligment among different compilers.
Sara
Sara
Hi all
Thanks for the help.
There seems to be a misunderstanding.
The DLL does exist, and I do have it's source. Now I am reponsible for it (just recently).
The DLL works on Windows platforms (32 bit). The users of this DLL (NOT me) have an h file, a lib file and the actual DLL. The users call the DLL's functions (and basically they are happy).
The example above was just an example (The actual structures have also shorts and chars).
Probably the packing issue was not thought of in the past. Now, I have came across one user who had decided to compile it's source with packing of 1 (for unknown reasons). The result was that "the DLL did not work".
Now I have no control over my users. They can do whatever they want (use any compiler the compile their source). I am looking for a way to prevent such future issues (Without major changes).
Thanks again.
Thanks for the help.
There seems to be a misunderstanding.
The DLL does exist, and I do have it's source. Now I am reponsible for it (just recently).
The DLL works on Windows platforms (32 bit). The users of this DLL (NOT me) have an h file, a lib file and the actual DLL. The users call the DLL's functions (and basically they are happy).
The example above was just an example (The actual structures have also shorts and chars).
Probably the packing issue was not thought of in the past. Now, I have came across one user who had decided to compile it's source with packing of 1 (for unknown reasons). The result was that "the DLL did not work".
Now I have no control over my users. They can do whatever they want (use any compiler the compile their source). I am looking for a way to prevent such future issues (Without major changes).
Thanks again.
as suggested you can add a size member into the struct which must be filled with sizeof(<structure>) by anyone and later can be checked when reading the structure.
many structures of WINAPI are made that way for example struct STARTUPINFO used for CreateProcess.
another method is to sort the member types by double, int, short, char and do all array sizes a multiple of 4 or add fillers (dummy members) so that each new type or array begins at a 32-bit offset.
that way even packed(1) structures should have same alignment what easily could be checked for one compiler.
Sara
many structures of WINAPI are made that way for example struct STARTUPINFO used for CreateProcess.
another method is to sort the member types by double, int, short, char and do all array sizes a multiple of 4 or add fillers (dummy members) so that each new type or array begins at a 32-bit offset.
that way even packed(1) structures should have same alignment what easily could be checked for one compiler.
Sara
>> The actual structures have also shorts and chars
To reduce the odds that packing will be a problem (this is not standard language talk and no guarantees), arrange the members from high width to low. So cluster all the doubles at the top, then floats, then longs, shorts, chars. Then add your own padding at the end so that the size of the struct is, say a multiple of 4 to 8 bytes long).
=====
When you provide a binary object, you also provide a contract w.r.t. the object. It includes the api, the installation and usage procedures. You may have forgotten to indicate your rules about compilation for usage with the DLL (e.g., do not pack).
For this one user, see if they will agree to modify their make file so that the modules that use your DLL they will not pack saying that the DLL requires no packing; and since it works, that is the way to handle this problem. If they insist, then you can offer to support their needs (up to you whether to charge) by customizing a special DLL (maybe with a name change or a folder change) that is packed and work with them to thoroughly test the new version.
For all other users, send them an amendment to the contract indicating the rules for compiling with usage of your DLL. They shouldn't mind since they are already working well with it.
Now you have two versions to maintain - one with and one without packing.
To reduce the odds that packing will be a problem (this is not standard language talk and no guarantees), arrange the members from high width to low. So cluster all the doubles at the top, then floats, then longs, shorts, chars. Then add your own padding at the end so that the size of the struct is, say a multiple of 4 to 8 bytes long).
=====
When you provide a binary object, you also provide a contract w.r.t. the object. It includes the api, the installation and usage procedures. You may have forgotten to indicate your rules about compilation for usage with the DLL (e.g., do not pack).
For this one user, see if they will agree to modify their make file so that the modules that use your DLL they will not pack saying that the DLL requires no packing; and since it works, that is the way to handle this problem. If they insist, then you can offer to support their needs (up to you whether to charge) by customizing a special DLL (maybe with a name change or a folder change) that is packed and work with them to thoroughly test the new version.
For all other users, send them an amendment to the contract indicating the rules for compiling with usage of your DLL. They shouldn't mind since they are already working well with it.
Now you have two versions to maintain - one with and one without packing.
Question has a verified solution.
Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.
Have a better answer? Share it in a comment.
Join & Write a Comment Already a member? Login.
Featured Post
Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.
Tackle projects and never again get stuck behind a technical roadblock.
Join Now