EddieH
asked on
problem storing struct into array
I need to read data from a file into an in-memory array but i'm unsure how to do it. The problem is i don't know how big the array is going to be. I store all the data into a structure as follows:
struct recData
{
char *surname;
char *firstname;
int age;
} RECORDS;
i succesfully tokenize the data into the required fields of my structure. But i am having problems storing them into my array.
I have a loop which goes through each line of the file and takes what i need.
I also am having problems mallocing the array, as i am unsure how something like that would be done as i have no idea how big my array is going to be.
Thanks for any help.
struct recData
{
char *surname;
char *firstname;
int age;
} RECORDS;
i succesfully tokenize the data into the required fields of my structure. But i am having problems storing them into my array.
I have a loop which goes through each line of the file and takes what i need.
I also am having problems mallocing the array, as i am unsure how something like that would be done as i have no idea how big my array is going to be.
Thanks for any help.
Since you don't know how many data items you will be storing, could you use something like a linked list instead of an array?
mr_septic is right. linked list is the best for this.
However, if you want to use array, that is posible too.
Just use array of pointers like this.
struct recData
{
char *surname;
char *firstname;
int age;
} RECORDS;
main()
{
RECORDS **rec_array;
int max, int num;
FILE *f;
/* open the file */
f = fopen(...);
// calculate number of records, add some for spare
max = (filelength(f) / sizeof(RECORDS)) + 10;
// malloc the array
rec_array = (RECORDS**) malloc(max * sizeof(RECORD*);
// now read it
for (num=0; !feof(f); num++) {
rec_array[num] = malloc(sizeof(RECORDS));
// read and assign the records to rec_array[num];
// I saw u are using char* for names, so you have to malloc it too !
.....
}
}
However, if you want to use array, that is posible too.
Just use array of pointers like this.
struct recData
{
char *surname;
char *firstname;
int age;
} RECORDS;
main()
{
RECORDS **rec_array;
int max, int num;
FILE *f;
/* open the file */
f = fopen(...);
// calculate number of records, add some for spare
max = (filelength(f) / sizeof(RECORDS)) + 10;
// malloc the array
rec_array = (RECORDS**) malloc(max * sizeof(RECORD*);
// now read it
for (num=0; !feof(f); num++) {
rec_array[num] = malloc(sizeof(RECORDS));
// read and assign the records to rec_array[num];
// I saw u are using char* for names, so you have to malloc it too !
.....
}
}
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks Kocil, that's what i need.
but i'm unsure how this line works:
max = (filelength(f) / sizeof(RECORDS)) + 10;
what is filelength?
but i'm unsure how this line works:
max = (filelength(f) / sizeof(RECORDS)) + 10;
what is filelength?
ASKER
by the way, i'm programming in ansi c
>>max = (filelength(f) / sizeof(RECORDS)) + 10
Kocil wrote this a pseudo code signifying that the length of the file divided by the size of each record will give you the number of records... hope I am right Kocil :-)
If you can get the number of records, then I don't think that you need to use an array of pointers. you can malloc for those many records !!!
If you will be accessing the data sequentially, linked list is a good option as mr_septic suggested. However, if the number of records is high, you will be doing a lot of mallocs which will slow down the program significantly.
better idea will be to malloc an array of records and keep adding records.
If you run out of memory while storing data you can use realloc() to increase the size of the allocated memory. Increase size by blocks of records (rather than a single record) to obtain better performance.
The data which you have assigned till then will not be affected if you are increasing the size of the allocated memory.
Kocil wrote this a pseudo code signifying that the length of the file divided by the size of each record will give you the number of records... hope I am right Kocil :-)
If you can get the number of records, then I don't think that you need to use an array of pointers. you can malloc for those many records !!!
If you will be accessing the data sequentially, linked list is a good option as mr_septic suggested. However, if the number of records is high, you will be doing a lot of mallocs which will slow down the program significantly.
better idea will be to malloc an array of records and keep adding records.
If you run out of memory while storing data you can use realloc() to increase the size of the allocated memory. Increase size by blocks of records (rather than a single record) to obtain better performance.
The data which you have assigned till then will not be affected if you are increasing the size of the allocated memory.
ASKER
thanks for the input. But i will be reading in an unknown number of records. All i know is that the records file i'm reading from is huge. In the order of a 100 megabytes or so. So, correct me if i'm wrong, but an array of pointers would be the most efficient way of getting all the data into an in-memory array. After i get everything in my array, i will be sorting them and writing them into another file. But i have no problem with all that and i have that all sorted out. The problem is reading in the data.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
thanks sunnycoder.
i'm not in front of my work computer at the moment so i assume that if i don't use all the allocated memory, i can use realloc to resize the array to better fit the amount of records. eg if i malloced 100,000 spaces and only used 80,000, i can use realloc to get rid of the excess malloced space? obviously i will be implementing a count to keep count of how many records i have.
can realloc be used in that manner?
i'm not in front of my work computer at the moment so i assume that if i don't use all the allocated memory, i can use realloc to resize the array to better fit the amount of records. eg if i malloced 100,000 spaces and only used 80,000, i can use realloc to get rid of the excess malloced space? obviously i will be implementing a count to keep count of how many records i have.
can realloc be used in that manner?
ASKER
ok, i have tried implementing Kocil's code into my program but i am getting seg faults. Here's the relevants parts of my code:
typedef struct recData
{
char *surname;
char *firstname;
int age;
}RECORDS;
RECORDS **rec_array;
RECORDS *wtemp;
/* opens files */
max = 50000; /* i just tried some arbitrary large number chosen at random */
rec_array = (RECORDS **)malloc(max * sizeof(RECORDS *));
memset(rec_array, 0, sizeof(rec_array));
/* get line from file in a while loop */
while(wtemp != NULL)
{
wtemp = (RECORDS *)malloc(sizeof(RECORDS));
wtemp = strtok(blah blah blah...)
/* tokenise the line and assign to variables first, second and age */
wtemp->firstname = first; /* same for the other values */
rec_array[num] = malloc(sizeof(RECORDS));
strcpy(rec_array[num]->fir
etc etc
the problem is the program only works for the first four lines of the input file, after which i get a seg fault. I've been staring at this code for ever now and i can't figure out why it's seg faulting. I have managed to figure out that the offending code is somewhere in the lines that i posted above by placing printf statements at various milestones. So the problem is definately around there somewhere.
thanks for any help
typedef struct recData
{
char *surname;
char *firstname;
int age;
}RECORDS;
RECORDS **rec_array;
RECORDS *wtemp;
/* opens files */
max = 50000; /* i just tried some arbitrary large number chosen at random */
rec_array = (RECORDS **)malloc(max * sizeof(RECORDS *));
memset(rec_array, 0, sizeof(rec_array));
/* get line from file in a while loop */
while(wtemp != NULL)
{
wtemp = (RECORDS *)malloc(sizeof(RECORDS));
wtemp = strtok(blah blah blah...)
/* tokenise the line and assign to variables first, second and age */
wtemp->firstname = first; /* same for the other values */
rec_array[num] = malloc(sizeof(RECORDS));
strcpy(rec_array[num]->fir
etc etc
the problem is the program only works for the first four lines of the input file, after which i get a seg fault. I've been staring at this code for ever now and i can't figure out why it's seg faulting. I have managed to figure out that the offending code is somewhere in the lines that i posted above by placing printf statements at various milestones. So the problem is definately around there somewhere.
thanks for any help
Nothing has happened on this question in more than 8 months. It's time for cleanup!
My recommendation, which I will post in the Cleanup topic area, is to
split points between Kocil and sunnycoder [grade B] (abandoned by experts after starting out well).
Please leave any comments here within the next four days.
PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!
jmcg
EE Cleanup Volunteer
My recommendation, which I will post in the Cleanup topic area, is to
split points between Kocil and sunnycoder [grade B] (abandoned by experts after starting out well).
Please leave any comments here within the next four days.
PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!
jmcg
EE Cleanup Volunteer