reading records from a binary file

I'm trying to read some records from a binary file.
The records are as such:
type(16 bits)
length(32 bits)
data[as specified by the length]

I used the binary file i/o function in conjunction with a structure for these 3 variables to no avail.
Its reading only junk values. any ideas as to how to implement this?
I'm thinking of using an int for type and long int for length [i'm not sure if it'll convert to int when it reads]. do i need to use an array for all 3 records??
gvashistAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

phoffric\Commented:
Again, as in your first question, I wrote: "if you take a stab at what you are trying to do, then one of us will try to give you the requested guidance. Post the code in the Code box, and if it does not work, then be sure to ask a specific question indicating what you expected and what went wrong."

Don't forget to include a comment that this is homework.

BTW, what OS and compiler (or IDE) are you using?
0
phoffric\Commented:
If you are self-studying, then to benefit from this process, it is important to show us what you have so that we can comment on your approach.
0
gvashistAuthor Commented:
here's what I have:

struct record
	{
		int type;
		long length;
        int data[]; 
	};

	int main()
	{
		FILE *ptr_tofile;
		struct record some_record;

		ptr_tofile=fopen("example.bin","rb");
		if (!ptr_tofile)
		{
			printf("Unable to open file!");
			return 1;
		}
		
        
			  
                      fread(&some_record,sizeof(int),1,ptr_myfile);
		      printf("%d\n",my_record.type);
			
                     fread(&some_record,sizeof(long),1,ptr_myfile);
		     printf("%d\n",my_record.length);
		     getch();
        }

Open in new window

0
Introduction to Web Design

Develop a strong foundation and understanding of web design by learning HTML, CSS, and additional tools to help you develop your own website.

phoffric\Commented:
>> int data[];
In a struct, to define an array of int's, you need an actual number of elements; so if you want 100 ints, you should write:
    int data[100];

After fixing this little problem, I found there are holes in the program - it does not compile due to undefined variables.

I recommend that you only post code that compiles. If you get a compiler error that you do not understand, then post that code, and ask for help.

When you post a program that produces incorrect results, you should indicate what you expected, and what you actually got.

0
gvashistAuthor Commented:
I have attached 2 files both of which compile and run.

now, i have restricted the i/o to just 2 elements: type and length.
i'm trying to write 2 blocks of data to get 32 bits for length but when I read, i manage to get only 16 bits worth of data.
how can I change that to write/read a 32 bit value.

also, how can i use the array to store data as long as dis ictated by the length variable

p.s: I like this process. I was a bit ignorant at the start. sorry about that.

thanks.
#include<stdio.h>
#include<conio.h>

	//reading the records I wrote
	struct rec
	{
		int type;
        long length;
        int data[100];
    };

	int main()
	{
		int count;
		FILE *ptr_tofile;
		struct rec some_record;

		ptr_tofile=fopen("Test.bin","rb");
		if (!ptr_tofile)
		{
			printf("Unable to open file!");
			return 1;
		}
		for ( count=1; count <= 10; count++)
		{
			fread(&some_record, sizeof(int), 1, ptr_tofile);
			printf("%d\n",some_record.type);
			fread(&some_record, sizeof(int), 2, ptr_tofile);
			printf("%d\n",some_record.type);
			getch();
			
		}
		fclose(ptr_tofile);
		return 0;
	}

Open in new window

#include<stdio.h>

//writing 10 records to binary file
	
	struct rec
	{
		int type;
        long length;
        int data[100];
    };

	int main()
	{
		int count;
		FILE *ptr_tofile;
		struct rec some_record;

		ptr_tofile=fopen("Test.bin","wb");
		if (!ptr_tofile)
		{
			printf("Unable to open file!");
			return 1;
		}
		for ( count=1; count <= 10; count++)
		{
			some_record.type= count;
			fwrite(&some_record, sizeof(int), 1, ptr_tofile);
			some_record.length=count;
			fwrite(&some_record, sizeof(int), 2, ptr_tofile);
			
		}
		fclose(ptr_tofile);
		return 0;
	}

Open in new window

0
phoffric\Commented:
Here are your two programs. I commented out the data array since you said you weren't using them (but you can add them back in). I've modified your write so that it writes an entire binary record out in one shot. With this approach, you read back one record at a time, and now you can print out the type and length in one print statement.
#include<stdio.h>  

//writing 10 records to binary file  

struct rec  
{  
    int type;  
    long length;  
    //int data[100];  
};  

int main()  
{  
    int count;  
    FILE *ptr_tofile;  
    struct rec some_record;  

    ptr_tofile=fopen("Test.bin","wb");  
    if (!ptr_tofile)  
    {  
        printf("Unable to open file!");  
        return 1;  
    }  
    for ( count=1; count <= 10; count++)  
    {  
        some_record.type   = count;
        some_record.length = count;
        fwrite(&some_record, sizeof(struct rec), 1, ptr_tofile);  
    }
    fclose(ptr_tofile);  
    return 0;  
}

Open in new window

0
phoffric\Commented:
Notice in the fwrite, that the second argument is the size of the entire structure. If you modify the structure, you do not have to change this fwrite statement. You are still writing the entire structure whether you increase or decrease the size of the structure.

size_t fwrite ( const void * ptr, size_t size, size_t count, FILE * stream );
    http://www.cplusplus.com/reference/clibrary/cstdio/fwrite/

"Return Value
The total number of elements successfully written is returned as a size_t object, which is an integral data type.
If this number differs from the count parameter, it indicates an error."


Always check for errors in all library calls when an error condition is detectable. That test remains to be done.
0
phoffric\Commented:
I only showed one program, since the read binary programj is just a mirror image of the first.
0
gvashistAuthor Commented:
Thank you.
It seems to work fine now.
Any idea how I could use the data array?
I want to read/write data as long as is specified in the length variable.
Can I use the same approach?
0
phoffric\Commented:
I see that you are new to EE. So, welcome to EE. Except perhaps for the simplest 2-3 line programs, we don't usually provide solutions. But once you show your level of expertise by posting what you have, we can probably point you in the right direction with a guided nudge.
0
phoffric\Commented:
>> I want to read/write data as long as is specified in the length variable.
>> Can I use the same approach?
If you don't mind that the file will have fixed length records but is only filled with some_record.length elements, then you just write the entire struct out, and read back one struct. Then upon reading a record, you can extract just the number of elements that you need from the array before overwriting it with the next record.

If you want to have the file to only contain exactly the variable number of elements in the struct, then you will need two writes and two reads. The first fwrite deals with just the record metadata consisting of just the type and length (you can make that a separate struct - a common way to handle Message or Record Headers).
0
phoffric\Commented:
I'll be back tomorrow to see if you have more questions, and look forward to a new post showing the addition of writing the array to the file.
0
gvashistAuthor Commented:
I managed to take a different approach to writing data. instead of writing it as an array, i decided to use the length(no. of bits of data) to write data in blocks.
I know it will be a problem if its not a multiple of 8 but thats a different problem.
right now, i'm trying to write type, length and a few blocks of data (short int)
The code is below, I'm writing the read program for the same. will this suffice??
#include<stdio.h>  

//writing 10 records to binary file  

struct rec  
{  
    int type;  
    long length;  
    short int data;  
};  

int main()  
{  
    int count,no_blocks;
    FILE *ptr_tofile;  
    struct rec some_record;  

    ptr_tofile=fopen("Test.bin","wb");  
    if (!ptr_tofile)  
    {  
        printf("Unable to open file!");  
        return 1;  
    }  
    for ( count=1; count <= 10; count++)  
    {  
        some_record.type   = count;
        some_record.length = (count*2);
      
           if(some_record.length<8) 
           {
                                    no_blocks=1;
           }
           else
           {
             no_blocks = some_record.length/sizeof(some_record.data);
           }
        some_record.data   = (count+1);
        fwrite(&some_record, sizeof(struct rec), 1, ptr_tofile);//writing type, length and one data block
        if(no_blocks>1)
        {  
           fwrite(&some_record.data, sizeof(some_record.data), no_blocks, ptr_tofile);//overwriting the 1st data block
        }
    }
    fclose(ptr_tofile);  
    return 0;  
}

Open in new window

0
phoffric\Commented:
Earlier, I suggested using a record header to include the record metadata. For example:
struct recordHdr {
    int type;  
    long length;
};
struct rec  
{  
    struct recordHdr;
    int data[100];  
};  

Open in new window


Before writing anything, you fill in one struct rec with both the metadata (i.e., the type and the length), and fill in "length" (e.g., 10) data elements. Now to write variable sized records to the disk, one way to do this is to use two writes, one for the metadata, and one for the data elements. The advice given here is very specific to this problem. If the data were something other than an array of plain old data types (POD), then there may be struct padding issues to deal with.

Now, suppose you have "length" data elements filled in. Then the 2nd write should have the number of elements set to "length". Give that a try.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
gvashistAuthor Commented:
Thanks Phoffric:

that works a lot better. I wrote some code to inculcate that and it seems to be working.
How different would this structure have to be if I'm reading binary data from a file. I mean, if the type, length and data is all binary. I know this might seem like an elementary question.
I'm not expecting any code. Just a clarification.
so, if the binary file already has a bunch of records with type, length and data all in binary, will I need to change anything?? the file should've stored everything in binary to begin with right?
0
phoffric\Commented:
In order to be meaningful, if you are going to try to write a number of data elements out, you should actually write values to the data. Your read program can then verify that the data is correct. In you code post, http:#35342434, you write out data which you never set.

I just noticed that in your OP, you refer to length
What is the units of length in your OP. It looks like it may be bits. But when writing out data to the disk fwrite refers to bytes.
    http://www.cplusplus.com/reference/clibrary/cstdio/fwrite/

If you need bits, then that leads to the question as to how you wish to represent, say 41 bits, and how you wish to write out a integral number of bytes to the disk.
0
phoffric\Commented:
If you post the code with your comments, that will help me understand better what you are trying to do. Your if statements in the http:#35342434 code and the usage of no_blocks is a concern for me; but that may be due to my not understanding what you are really trying to do. The big concern was that you were writing data bytes out that you hadn't even set.
0
gvashistAuthor Commented:
yes, I fixed that. I now have actual data being written into the file.

I'm considering a situation where I'm already armed with a file where type and length are in bits (16 and 32 respectively) followed by data.
If the type and length are in bits, can I read them into an int or long int directly?
or do I need to convert them first?
can I have an array of binary data which I assume is what I'd need in this case?
0
phoffric\Commented:
When you use fwrite, then number of bits have to be converted to number of bytes. If your application always has the number of bits as an integral multiple of 8, then why not just use number of bytes and not deal with bits?

If you know the max number of data elements that you will need to be written in one record, then you can define the struct using that hard-coded number.

If you do not know the max number of data elements that can exist in one record, then you will have to dynamically allocate an array to handle the unknown sizes as the program discovers this amount. To do this, you can use malloc and free.
   http://www.cplusplus.com/search.do?q=malloc
   http://www.cplusplus.com/reference/clibrary/cstdlib/free/
0
gvashistAuthor Commented:
the length variable is supposed to give me the no of data bits. (32 bits)
so, max. number of data bits = 2^32. can I have a pointer to an array that contains binary data?
If the number of data bits is always a multiple of 8 and I know max. number of data elements (length/8), then, is the program I wrote here valid for read? considering of course the fwrite is replaced with fread and there is already data in the bin file.
0
phoffric\Commented:
>> the length variable is supposed to give me the no of data bits. (32 bits)
>> so, max. number of data bits = 2^32
Well that is one way to define length. Usually, we define length in terms of bytes, or even a higher unit.

>>  can I have a pointer to an array that contains binary data?
yes.

>> If the number of data bits is always a multiple of 8 and I know max. number of data elements (length/8), then, is the program I wrote here valid for read?
    You'll have to post the read and write programs for comments.
    Your read program should have a verfication test in it. Since you know what you write, you can confirm that all that was written is correctly read in properly.
0
Dirk HaestProject managerCommented:
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.