Solved

How to read structure from binary file

Posted on 2001-08-24
11
329 Views
Last Modified: 2008-03-06


If you have a large structure made up of different types (e.g. int, char, float, enum, etc), sub-structures,
pointers, arrays and single elements stored in a binary file what is the best/portable way of  (saving and)
reading it into a structure, when you don't know the size of the structure data in the binary file beforehand.

To save the structure I did a fwrite like

fwrite ((void*) &ptr_structure,1, bytes_to_write, (FILE *)ptr_file);


It seems to have worked. An output file was created and when I viewed the binary file at least the right strings
were in there.

But I can't seem to be able to read the file and repopulate the structure. I can't use a fread since I
won't know the size of the structure data in the file, so I used a fgetc() and tried to read it character by character using

while ( !feof(file_handle)
{
  *ptr_structure = (char)fgetc(file_handle);
}

Although it doesn't report an error, when I tried to find the earlier strings they weren't where they were supposed
to be, or anywhere else? Am I missing some form of alignment? Or is the fwrite I did wrong in the first place.

According to the C FAQ (http://www.faqs.org/faqs/C-faq/faq/) (Question 2.11) using a fwrite means the file
is not portable, in particular pointer information is lost or distorted.  Also, (when the size of the structure
is not known) the only reliable way of reading in a structure is element by element.  Are these really the
case?  What do people do when they have a large structure?

FlipFlop

0
Comment
Question by:FlipFlop
  • 5
  • 5
11 Comments
 
LVL 47

Expert Comment

by:dbrunton
ID: 6421126
This is one comment.  Others will comment as well.

You can use a limited number of types in your structure for portability.  In the case of pointers dont' use them because you don't know what system your app will be running on and how pointers are defined there.  You may also not know what order that numbers are handled, big endian or little endian.  This is dependent on how the processor works.

I think C has the sizeof function so you should be able to use it on your structure and get it's size that way.
0
 
LVL 1

Expert Comment

by:stewe
ID: 6421346

You should use a header in your saved file. You can store the length (or the type of the item), so the reading routine can restore it correctly.

   Stewe
0
 

Author Comment

by:FlipFlop
ID: 6421402
I've thought of using a header too.  I was thinking along the lines

1. When saving, using fwrite(), put the size of the structure in, say, the first X bytes.
2. When reading, read the first X bytes, get the size of the structure then do a fread(...,sizeof(X),...)

Is this what you meant?


FlipFlop
0
 

Author Comment

by:FlipFlop
ID: 6421530

Also, can you explain why the fgetc() used as

while ( !feof(file_handle)
{
 *ptr_structure = (char)fgetc(file_handle);
}

fails to read the structure?  I even tried it with a structure without any pointers, so only sub-structures, arrays and single elements of chars, ints, floats, enums but it still didn't work.  How does not fgetc() work?

FlipFlop
0
 
LVL 1

Expert Comment

by:stewe
ID: 6421542

With *ptr_structure, you update the first byte.
Try something like this:

char *p=(char *)&ptr_structure;
while ( !feof(file_handle)
{
  *p++ = (char)fgetc(file_handle);
}

   Stewe
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:FlipFlop
ID: 6421665
Hi,

I tried a simple example to test this, using a win32 console application.

******************************************************
// TestStructRead.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include "stdio.h"
#include "stdlib.h"

#define      NUM     4

struct details{
     char     name[1024];
     int          age;
     float     weight;
     float     height;
}     people[NUM];

struct rdetails{
     char     name[1024];
     int          age;
     float     weight;
     float     height;
}     rpeople[NUM];


int main(int argc, char* argv[])
{

     FILE     *fp1;
     FILE     *fp2;
     int          i;

     // Prepare the structure to write to file
     printf( "Input a name: " );
     gets( people[0].name );
//     people[1].name          = "David";
     people[0].age          = 10;
     people[0].weight     = 100.0;
     people[0].height     = 150.0;

     printf( "Input a name: " );
     gets( people[1].name );
//     people[1].name          = "Jon";
     people[1].age          = 15;
     people[1].weight     = 223.0;
     people[1].height     = 95.0;

     printf( "Input a name: " );
     gets( people[2].name );
//     people[2].name          = "Howard";
     people[2].age          = 35;
     people[2].weight     = 563.0;
     people[2].height     = 987.0;

     printf( "Input a name: " );
     gets( people[3].name );
//     people[3].name          = "Steve";
     people[3].age          = 111;
     people[3].weight     = 786.0;
     people[3].height     = 98.0;

     // Open file for writing
     if ((fp1 = fopen("data","wb")) == NULL)
     {
          printf("Error: Cannot open file data for binary write\n");
          exit(0);
     }

     // Write data to file
     if (fwrite(people,sizeof(people),1,fp1) != 1)
     {
          printf("Error: Cannot save data to file\n");
          exit(0);
     }


     // Open file for reading
     if ((fp2 = fopen("data","rb")) == NULL)
     {
          printf("Error: Cannot open file data for binary read\n");
          exit(0);
     }

     // Read data into another structure
     char *p=(char *)rpeople;
     while ( !feof(fp2))
     {
          *p++ = (char)fgetc(fp2);
     }

     // Print out data
     printf("\n************************************\n");
     for (i=0;i<NUM;i++)
     {
          printf("Name: \t\t%s\n",rpeople[i].name);
          printf("Age: \t\t%d\n",rpeople[i].age);
          printf("Weight:  \t%f\n",rpeople[i].weight);
          printf("Height:  \t%f\n",rpeople[i].height);
     }
     printf("************************************\n");

     return 0;
}

******************************************************

When I ran it I noticed that age, weight and height for the last item is always zero.  When I changed NUM to 3,
all the output printf's produced zero?


Err, FlipFlop


0
 
LVL 1

Expert Comment

by:stewe
ID: 6421682

I was wrong, examining your code.
For reading:

for reading:

fread(&rpeople,sizeof(rpeople),1,fp2);


   Stewe
0
 

Author Comment

by:FlipFlop
ID: 6421773
Hi,

Doesn't seem to make any difference.  The last item,
rpeople[3], still prints out

Name:  Steve
Age:   0
Weight:  0.000
Height:  0.000

Does your's do this too?  Although both the fread and
your earlier fgetc() both appear to be reading the rest
of the data from file correctly.  So what was wrong with
your earlier suggestion.


FlipFlop



0
 
LVL 1

Expert Comment

by:stewe
ID: 6421883

Ok, we got flushing problems.
Because 4*1036=4144, 1036 is the sizeof(details), and the file buffer size is 4096, so only 4096 was written.
Put an fflush(fp1) after writing, then reading will work.

  Stewe
0
 
LVL 1

Accepted Solution

by:
stewe earned 40 total points
ID: 6421890

Better reading & writing:

for(int n=0;n<NUM;n++)
  fwrite(&(people[n]),sizeof(details),1,fp1);
fflush(fp1);
fclose(fp1);

if ((fp2 = fopen("data","rb")) == NULL)
{
  printf("Error: Cannot open file data for binary read\n");
  exit(0);
}

for(int n=0;n<NUM;n++)
  fread(&(rpeople[n]),sizeof(rdetails),1,fp2);
fclose(fp2);


  Stewe
0
 

Author Comment

by:FlipFlop
ID: 6421968
Hi,

Yep everything working fine now.  Even when I put the
fgetc() lines in.  Now I am confused.  

I've released the points, but if you do manage to figure out why the fgetc() fails I'd be really interested to know.

TTFN, FlipFlop


0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

This article is meant to give a basic understanding of how to use R Sweave as a way to merge LaTeX and R code seamlessly into one presentable document.
A short article about problems I had with the new location API and permissions in Marshmallow
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now