Solved

packing & bit fields

Posted on 2000-02-19
13
281 Views
Last Modified: 2010-08-05
I have a structure which contains elements of various sizes - one of which is a bitfield containing elements that are 1, 2 and 3 bits wide.

The code compiled and worked fine as a Win32 console program, but under Linux the packing/bitfields are all weird and nothing lines up properly.

I know about the PACKED keyword to use in the structure definition - but it seems to make little difference.

Anyone got any ideas?
0
Comment
Question by:shivers
  • 6
  • 6
13 Comments
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
How about letting us see some relevant code snippits of what you are doing.
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
Adjusted points to 75
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
Good idea - dunno why i didn't in the first place - here's one of the structs (this one for a GIF file header):

struct PACKED {
      char ID[6];
      word width;
      word height;
      struct {
            unsigned bpp : 3;
            unsigned reserved : 1;
            unsigned color_res : 3;
            unsigned extra : 1;
      } bm;
      byte background;
      byte reserved;
} hdr;

And the data is read from the file with a simple

fread(&hdr, sizeof(hdr), 1, datafile);

The width and height values work ok - but the bpp, color_res, etc don't at all.
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
forgot to say, word, dword and byte are #defined as:

word  unsigned short int
dword unsigned long int
byte  unsigned char
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Okay, the struct definition is valid. Lets see what ANSI says about bit fields (as described in "C A Reference Manual" third edition):

"The precise manner in which components (and especially bit fields) are packed into a structure is implementation-dependent but is predictable for each implementation. The intent is that bit fields should be packed as tightly as possible in a structure... The use of bitfields is therefore likely to be nonportable"

The "PACKED" attribute simply instructs the compiler that (from Gcc docs):

"This attribute, attached to an enum, struct, or union type definition, specified that the minimum required memory be used to represent the type."

The results of which are also implementation dependent. So, my guess is that you are reading data on one platform that was written on another. According to what I see there is no assurance that this can be done in this manner.
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
eh?  Does that mean there's no way at all to have real bit fields under Linux?

Surely there has to be some way of doing this - otherwise it's back to bit shifts and AND operations to get the data out.

0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Bit fields work great, they just aren't portable between between different implementations of the compiler and/or hardware platforms. If you are only working with one platform/compiler they are a good means of reducing storage and compute resources.However, if the data and application needs to be platform independant it's bit operations. Oh yeah, you also have to watch out for word size and little/big endian differences.
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
Ok - so assuming I just want my code to work on RedHat Linux 6 (i386) and I'm compiling with gcc 2.91.66 how do I get them to work?

0
 
LVL 3

Expert Comment

by:alien_life_form
Comment Utility
Greetings.

Assuming all you want is reading GIFs,
my advice would be reading the code of a gif manipulation package - libungif
comes to mind. There you'll find (presumably portable) ways of doing what you want.

Relying on structure layouts in memory is almost never going to behave the way you expect when you switch platforms, operating systems or even just compilers...

Cheers,

  alf
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
Since you mentioned win32 apps, I sounds sort of like you're wanting the same code to be able to run on more than one platform. If this is the case then you'd probably want to either extract the gif reader code from one of the gif libraries or applications. There's source for a couple at ftp://prtr-13.ucsc.edu/pub/libungif/.

Although the use of gif is pretty wide spread, they've somewhat fallen out of favor, primarily do to legal issues regarding the Unisys patent. If you're not tightly tied to that format, you might consider PNG, which is a more compact format with full compression capabilities. There are platform independant libraries and tools for PNG format (http://www.cdrom.com/pub/png/ http://www.boutell.com/gd/) and others.
0
 
LVL 1

Author Comment

by:shivers
Comment Utility
Your input is appreciated, but my concerns lie not just with GIF files, but (essentially) any and every binary file format.  The modules I am writing will form a part of my final project for my degree - so I want to avoid using code written by someone else wherever possible.

Cross platform compatibility is midly important - but getting it working under Linux is my primary concern (writing platform dependant versions I will leave for now).
0
 
LVL 40

Expert Comment

by:jlevie
Comment Utility
I'm looking at you last comment and I'm trying to figure out exactly what you mean by "not just with GIF files, but (essentially) any and every binary file format". That covers a vast range of possibilities and there isn't any single answer. Some binary file formats are intended to be platform independent and the file format specifies what the data form is (big endian, little endian), floating point encoding format, etc. Other binary file formats aren't platform indepependent and you have to figure out for yourself how the data is laid in the file and what operations are necessary to convert the non-native binary data.

Now if the application that reads the data is the same application that writes the data (uni-platform), none of the above applies. As long as the data structure definitions are the same and the write/read operations are symetrical everything will line up.

The general solution for dealing with a bit fields is either isolate the bit(s) of interest with masks and then shift if necessary. If, for example, the binary file format defined a byte that contained the data:

bbbfcccr  (3 bits for bpp, a flag bit, 3 color bits, and a reserved bit)

I'd have three masks to isolate the data of interest (0xE0, 0x10, & 0x0E), which I'd apply as masks and shift the result into the proper position, e.g:

unsigned char raw, bpp, msb_first, col;
msb_first = (raw&0x10)>>4;
bpp = (raw&0xe0)>>5;
col = (raw&0x0e)>>1;


0
 
LVL 40

Accepted Solution

by:
jlevie earned 75 total points
Comment Utility
See comment of 02/20/00 and 02/21/00 for the answer
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Linux Offline Package 3 311
pipe to sed or perl, please help 5 120
usage of %{name} in rpm spec file 4 49
Problem to start Neon 20 48
Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
This video discusses moving either the default database or any database to a new volume.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now